We study connections between differential equations and optimization algorithms for $m$-strongly and $L$-smooth convex functions through the use of Lyapunov functions by generalizing the Linear Matrix Inequality framework developed by Fazylab et al. in 2018. Using the new framework we derive analytically a new (discrete) Lyapunov function for a two-parameter family of Nesterov optimization methods and characterize their convergence rate. This allows us to prove a convergence rate that improves substantially on the previously proven rate of Nesterov's method for the standard choice of coefficients, as well as to characterize the choice of coefficients that yields the optimal rate. We obtain a new Lyapunov function for the Polyak ODE and revisit the connection between this ODE and the Nesterov's algorithms. In addition discuss a new interpretation of Nesterov method as an additive Runge-Kutta discretization and explain the structural conditions that discretizations of the Polyak equation should satisfy in order to lead to accelerated optimization algorithms.
翻译:我们通过推广Fazylab等人于2018年建立的线性矩阵不等式框架,利用Lyapunov函数研究微分方程与针对 $m$-强凸且$L$-光滑凸函数的优化算法之间的联系。借助该新框架,我们解析地推导出一类双参数Nesterov优化方法的全新(离散)Lyapunov函数,并刻画其收敛速率。这使我们能够证明:对于标准系数选择,所证收敛速率较此前Nesterov方法的速率有显著提升,同时可刻画达到最优速率的系数选取条件。我们为Polyak常微分方程获得了新的Lyapunov函数,并重新审视了该常微分方程与Nesterov算法之间的关联。此外,本文讨论了将Nesterov方法解释为加性Runge-Kutta离散格式的新视角,并阐明了Polyak方程离散格式为获得加速优化算法所应满足的结构性条件。