Though quasi-Newton methods have been extensively studied in the literature, they either suffer from local convergence or use a series of line searches for global convergence which is not acceptable in the distributed setting. In this work, we first propose a line search free greedy quasi-Newton (GQN) method with adaptive steps and establish explicit non-asymptotic bounds for both the global convergence rate and local superlinear rate. Our novel idea lies in the design of multiple greedy quasi-Newton updates, which involves computing Hessian-vector products, to control the Hessian approximation error, and a simple mechanism to adjust stepsizes to ensure the objective function improvement per iterate. Then, we extend it to the master-worker framework and propose a distributed adaptive GQN method whose communication cost is comparable with that of first-order methods, yet it retains the superb convergence property of its centralized counterpart. Finally, we demonstrate the advantages of our methods via numerical experiments.

翻译：尽管拟牛顿方法在文献中已被广泛研究，但它们要么存在局部收敛问题，要么为了实现全局收敛而采用一系列线性搜索，这在分布式设置中是不可接受的。本文首先提出一种无需线性搜索的自适应步长贪心拟牛顿（GQN）方法，并建立了全局收敛速率和局部超线性速率两者的显式非渐近界。我们的创新之处在于设计了多种贪心拟牛顿更新（涉及计算Hessian-向量乘积）以控制Hessian近似误差，以及一种简单的步长调整机制来确保每次迭代中目标函数的改进。随后，我们将该方法扩展到主从（master-worker）框架，提出一种分布式自适应GQN方法，其通信成本与一阶方法相当，同时保留了集中式版本的卓越收敛性质。最后，通过数值实验验证了所提方法的优势。