Despite the impressive numerical performance of quasi-Newton and Anderson/nonlinear acceleration methods, their global convergence rates have remained elusive for over 50 years. This paper addresses this long-standing question by introducing a framework that derives novel and adaptive quasi-Newton or nonlinear/Anderson acceleration schemes. Under mild assumptions, the proposed iterative methods exhibit explicit, non-asymptotic convergence rates that blend those of gradient descent and Cubic Regularized Newton's method. Notably, these rates are achieved adaptively, as the method autonomously determines the optimal step size using a simple backtracking strategy. The proposed approach also includes an accelerated version that improves the convergence rate on convex functions. Numerical experiments demonstrate the efficiency of the proposed framework, even compared to a fine-tuned BFGS algorithm with line search.

翻译：尽管拟牛顿法和安德森/非线性加速方法在数值计算中表现卓越，但其全局收敛速率问题在长达50余年的研究中仍悬而未决。本文针对这一长期存在的难题，提出了一种新框架，由此衍生出兼具自适应性与创新性的拟牛顿或非线性/安德森加速方案。在温和假设条件下，所提出的迭代方法展现出显式非渐近收敛速率，该速率融合了梯度下降法与三次正则牛顿法的特性。尤为重要的是，该速率通过自适应方式实现——方法采用简单回溯策略自主确定最优步长。本文还提出了加速变体，可提升凸函数上的收敛速率。数值实验表明，即使与经过精细调优且带线性搜索的BFGS算法相比，本框架仍展现出卓越的计算效率。