Stochastic gradient descent (SGD) is a promising method for solving large-scale inverse problems, due to its excellent scalability with respect to data size. The current mathematical theory in the lens of regularization theory predicts that SGD with a polynomially decaying stepsize schedule may suffer from an undesirable saturation phenomenon, i.e., the convergence rate does not further improve with the solution regularity index when it is beyond a certain range. In this work, we present a refined convergence rate analysis of SGD, and prove that saturation actually does not occur if the initial stepsize of the schedule is sufficiently small. Several numerical experiments are provided to complement the analysis.

翻译：由于在数据大小方面具有极好的伸缩性,因此,Stochastemic Temple Spolition(SGD)是解决大规模反向问题的一个很有希望的方法,目前从正规化理论角度看的数学理论预测,具有多子化步骤化时间表的SGD可能会受到不可取的饱和现象的影响,即当溶性常规指数超出某一范围时,趋同率不会随着溶性常规指数而进一步改善。在这项工作中,我们提出了对SGD的精细的趋同率分析,并证明如果该时间表的初步逐步化规模足够小,则实际不会出现饱和。提供了数项实验来补充分析。