Quasi-Newton algorithms are among the most popular iterative methods for solving unconstrained minimization problems, largely due to their favorable superlinear convergence property. However, existing results for these algorithms are limited as they provide either (i) a global convergence guarantee with an asymptotic superlinear convergence rate, or (ii) a local non-asymptotic superlinear rate for the case that the initial point and the initial Hessian approximation are chosen properly. In particular, no current analysis for quasi-Newton methods guarantees global convergence with an explicit superlinear convergence rate. In this paper, we close this gap and present the first globally convergent quasi-Newton method with an explicit non-asymptotic superlinear convergence rate. Unlike classical quasi-Newton methods, we build our algorithm upon the hybrid proximal extragradient method and propose a novel online learning framework for updating the Hessian approximation matrices. Specifically, guided by the convergence analysis, we formulate the Hessian approximation update as an online convex optimization problem in the space of matrices, and we relate the bounded regret of the online problem to the superlinear convergence of our method.

翻译：拟牛顿算法是求解无约束最小化问题最流行的迭代方法之一，主要归功于其优越的超线性收敛性质。然而，现有关于这些算法的结果存在局限性，它们要么提供具有渐近超线性收敛速率的全局收敛保证，要么仅针对初始点和初始Hessian矩阵近似选择恰当的情形给出局部非渐近超线性收敛速率。特别是，目前尚无针对拟牛顿方法的分析能够保证具有显式超线性收敛速率的全局收敛性。在本文中，我们填补了这一空白，首次提出了一种具有显式非渐近超线性收敛速率的全局收敛拟牛顿方法。与经典拟牛顿方法不同，我们的算法基于混合邻近外梯度方法构建，并提出了一种新颖的在线学习框架来更新Hessian近似矩阵。具体而言，在收敛分析的指导下，我们将Hessian近似更新问题转化为矩阵空间中的在线凸优化问题，并将在线问题的有界遗憾与方法的超线性收敛性相关联。