In this paper, we propose a Dimension-Reduced Second-Order Method (DRSOM) for convex and nonconvex (unconstrained) optimization. Under a trust-region-like framework, our method preserves the convergence of the second-order method while using only curvature information in a few directions. Consequently, the computational overhead of our method remains comparable to the first-order such as the gradient descent method. Theoretically, we show that the method has a local quadratic convergence and a global convergence rate of $O(\epsilon^{-3/2})$ to satisfy the first-order and second-order conditions if the subspace satisfies a commonly adopted approximated Hessian assumption. We further show that this assumption can be removed if we perform a corrector step using a Krylov-like method periodically at the end stage of the algorithm. The applicability and performance of DRSOM are exhibited by various computational experiments, including $L_2 - L_p$ minimization, CUTEst problems, and sensor network localization.

翻译：本文提出了一种适用于凸与非凸（无约束）优化的降维二阶方法（DRSOM）。在类似信赖域的框架下，我们的方法在仅利用少数方向的曲率信息的同时，保留了二阶方法的收敛性。因此，该方法的计算开销仍与梯度下降法这类一阶方法相当。理论上，我们证明：若子空间满足常用的近似海森矩阵假设，该方法具有局部二次收敛性，且为满足一阶与二阶条件，全局收敛率为$O(\epsilon^{-3/2})$。我们进一步证明，若在算法末期定期采用类Krylov方法执行校正步骤，则可移除该假设。通过包括$L_2 - L_p$最小化、CUTEst问题及传感器网络定位在内的多种计算实验，展示了DRSOM的适用性与性能。