We consider solving equality-constrained nonlinear, nonconvex optimization problems. This class of problems appears widely in a variety of applications in machine learning and engineering, ranging from constrained deep neural networks, to optimal control, to PDE-constrained optimization. We develop an adaptive inexact Newton method for this problem class. In each iteration, we solve the Lagrangian Newton system inexactly via a randomized iterative sketching solver, and select a suitable stepsize by performing line search on an exact augmented Lagrangian merit function. The randomized solvers have advantages over deterministic linear system solvers by significantly reducing per-iteration flops complexity and storage cost, when equipped with suitable sketching matrices. Our method adaptively controls the accuracy of the randomized solver and the penalty parameters of the exact augmented Lagrangian, to ensure that the inexact Newton direction is a descent direction of the exact augmented Lagrangian. This allows us to establish a global almost sure convergence. We also show that a unit stepsize is admissible locally, so that our method exhibits a local linear convergence. Furthermore, we prove that the linear convergence can be strengthened to superlinear convergence if we gradually sharpen the adaptive accuracy condition on the randomized solver. We demonstrate the superior performance of our method on benchmark nonlinear problems in CUTEst test set, constrained logistic regression with data from LIBSVM, and a PDE-constrained problem.

翻译：本文考虑求解等式约束的非线性、非凸优化问题。这类问题广泛出现在机器学习和工程领域的各类应用中，涵盖从约束深度神经网络、最优控制到偏微分方程约束优化等场景。我们针对此类问题开发了一种自适应非精确牛顿法。在每次迭代中，我们通过随机迭代草图求解器非精确地求解拉格朗日牛顿系统，并通过在精确增广拉格朗日价值函数上进行线搜索来选择合适的步长。当配备适当的草图矩阵时，随机求解器相较于确定性线性系统求解器具有显著优势：可大幅降低每次迭代的浮点运算复杂度与存储成本。我们的方法通过自适应控制随机求解器的精度和精确增广拉格朗日的惩罚参数，确保非精确牛顿方向为精确增广拉格朗日函数的下降方向，从而建立了全局几乎必然收敛性。我们还证明单位步长在局部范围内可接受，因此该方法具有局部线性收敛性。进一步地，若逐步增强随机求解器的自适应精度条件，可证明线性收敛可强化为超线性收敛。我们在CUTEst测试集的基准非线性问题、LIBSVM数据集的约束逻辑回归问题以及偏微分方程约束问题上验证了该方法的优异性能。