We analyze Newton's method with lazy Hessian updates for solving general possibly non-convex optimization problems. We propose to reuse a previously seen Hessian for several iterations while computing new gradients at each step of the method. This significantly reduces the overall arithmetical complexity of second-order optimization schemes. By using the cubic regularization technique, we establish fast global convergence of our method to a second-order stationary point, while the Hessian does not need to be updated each iteration. For convex problems, we justify global and local superlinear rates for lazy Newton steps with quadratic regularization, which is easier to compute. The optimal frequency for updating the Hessian is once every $d$ iterations, where $d$ is the dimension of the problem. This provably improves the total arithmetical complexity of second-order algorithms by a factor $\sqrt{d}$.

翻译：我们分析了采用惰性海森矩阵更新的牛顿法，用于求解一般可能的非凸优化问题。我们提出在方法的每一步计算新梯度的同时，将先前计算的海森矩阵重复使用若干次。这显著降低了二阶优化方案的总体算术复杂度。通过采用三次正则化技术，我们证明了该方法能快速全局收敛至二阶稳定点，且无需在每个迭代步骤更新海森矩阵。对于凸问题，我们验证了采用二次正则化（其计算更为简便）的惰性牛顿步具有全局和局部超线性收敛速度。海森矩阵的最优更新频率为每$d$次迭代一次，其中$d$为问题维度。这使二阶算法的总算术复杂度得到了因子$\sqrt{d}$的确定性改进。