We propose a Randomized Progressive Training algorithm (RPT) -- a stochastic proxy for the well-known Progressive Training method (PT) (Karras et al., 2017). Originally designed to train GANs (Goodfellow et al., 2014), PT was proposed as a heuristic, with no convergence analysis even for the simplest objective functions. On the contrary, to the best of our knowledge, RPT is the first PT-type algorithm with rigorous and sound theoretical guarantees for general smooth objective functions. We cast our method into the established framework of Randomized Coordinate Descent (RCD) (Nesterov, 2012; Richt\'arik & Tak\'a\v{c}, 2014), for which (as a by-product of our investigations) we also propose a novel, simple and general convergence analysis encapsulating strongly-convex, convex and nonconvex objectives. We then use this framework to establish a convergence theory for RPT. Finally, we validate the effectiveness of our method through extensive computational experiments.

翻译：我们提出了一种随机渐进式训练算法（RPT）——对著名的渐进式训练方法（PT）（Karras等人，2017）的随机近似。PT最初设计用于训练生成对抗网络（GAN）（Goodfellow等人，2014），但被提出时仅是一种启发式方法，即便在最简单的目标函数上也没有收敛性分析。相反，据我们所知，RPT是首个在一般光滑目标函数上具有严格且可靠理论保证的PT类算法。我们将该方法纳入随机坐标下降（RCD）（Nesterov，2012；Richtárik & Takáč，2014）的已有框架中，并在此框架下（作为我们研究的副产品）提出了一种新颖、简单且通用的收敛性分析，该分析涵盖了强凸、凸和非凸目标函数。随后，我们利用该框架建立了RPT的收敛理论。最后，通过大量计算实验验证了我们方法的有效性。