This work establishes provably faster convergence rates for gradient descent via a computer-assisted analysis technique. Our theory allows nonconstant stepsize policies with frequent long steps potentially violating descent by analyzing the overall effect of many iterations at once rather than the typical one-iteration inductions used in most first-order method analyses. We show that long steps, which may increase the objective value in the short term, lead to provably faster convergence in the long term. A conjecture towards proving a faster $O(1/T\log T)$ rate for gradient descent is also motivated along with simple numerical validation.

翻译：摘要：本文通过计算机辅助分析技术，建立了梯度下降算法可证明的更优收敛速率。我们的理论允许采用非恒定步长策略，通过一次性分析多个迭代的整体效应（而非典型一阶方法分析中常用的单步归纳），允许频繁出现可能违反下降特性的长步长。研究表明，短期内可能增加目标函数值的长步长，在长期内可带来可证明的更快收敛速度。此外，本文提出了一种关于证明梯度下降达到 $O(1/T\log T)$ 更快速率的猜想，并辅以简单的数值验证。