We consider unconstrained minimization of smooth convex functions. We propose a novel variational perspective using forced Euler-Lagrange equation that allows for studying high-resolution ODEs. Through this, we obtain a faster convergence rate for gradient norm minimization using Nesterov's accelerated gradient method. Additionally, we show that Nesterov's method can be interpreted as a rate-matching discretization of an appropriately chosen high-resolution ODE. Finally, using the results from the new variational perspective, we propose a stochastic method for noisy gradients. Several numerical experiments compare and illustrate our stochastic algorithm with state of the art methods.
翻译:我们考虑光滑凸函数的无约束极小化问题。通过引入受迫欧拉-拉格朗日方程,我们提出了一种新颖的变分视角,从而能够研究高分辨率ODE。基于此,我们利用Nesterov加速梯度方法获得了梯度范数最小化的更快收敛速率。此外,我们证明Nesterov方法可以视为对适当选取的高分辨率ODE进行速率匹配离散化的结果。最终,基于新变分视角的结论,我们针对含噪梯度提出了一种随机方法。多项数值实验将我们的随机算法与当前最优方法进行了比较和展示。