The high-resolution differential equation framework has been proven to be tailor-made for Nesterov's accelerated gradient descent method~(\texttt{NAG}) and its proximal correspondence -- the class of faster iterative shrinkage thresholding algorithms (FISTA). However, the systems of theories is not still complete, since the underdamped case ($r < 2$) has not been included. In this paper, based on the high-resolution differential equation framework, we construct the new Lyapunov functions for the underdamped case, which is motivated by the power of the time $t^{\gamma}$ or the iteration $k^{\gamma}$ in the mixed term. When the momentum parameter $r$ is $2$, the new Lyapunov functions are identical to the previous ones. These new proofs do not only include the convergence rate of the objective value previously obtained according to the low-resolution differential equation framework but also characterize the convergence rate of the minimal gradient norm square. All the convergence rates obtained for the underdamped case are continuously dependent on the parameter $r$. In addition, it is observed that the high-resolution differential equation approximately simulates the convergence behavior of~\texttt{NAG} for the critical case $r=-1$, while the low-resolution differential equation degenerates to the conservative Newton's equation. The high-resolution differential equation framework also theoretically characterizes the convergence rates, which are consistent with that obtained for the underdamped case with $r=-1$.
翻译:高分辨率微分方程框架已被证明是为Nesterov加速梯度下降方法(\texttt{NAG})及其近端对应算法——快速迭代收缩阈值算法(FISTA)类量身定制的。然而,该理论体系尚不完整,因为阻尼不足情形($r < 2$)尚未被纳入。本文基于高分辨率微分方程框架,受混合项中时间$t^{\gamma}$或迭代次数$k^{\gamma}$的幂次启发,为阻尼不足情形构造了新的Lyapunov函数。当动量参数$r$为$2$时,新Lyapunov函数与先前结果一致。这些新证明不仅涵盖了先前基于低分辨率微分方程框架获得的目标值收敛速率,还刻画了最小梯度范数平方的收敛速率。对于阻尼不足情形获得的所有收敛速率均连续依赖于参数$r$。此外,研究发现高分辨率微分方程能够近似模拟临界情形$r=-1$下\texttt{NAG}的收敛行为,而低分辨率微分方程则退化为保守牛顿方程。高分辨率微分方程框架还从理论上表征了收敛速率,其结果与$r=-1$阻尼不足情形获得的结论一致。