Dynamic sparse training algorithms have shown promise in achieving high performance while reducing resource costs, making them an attractive option in machine learning. However, despite their potential, the theoretical properties of dynamic sparse training remain largely unexplored. My research aims to fill this knowledge gap by investigating the theoretical properties of sparse training models. As a result, guidelines will be developed for applying dynamic sparse training to real-world problems.