We study nonlinear optimization problems with a stochastic objective and deterministic equality and inequality constraints, which emerge in numerous applications including finance, manufacturing, power systems and, recently, deep neural networks. We propose an active-set stochastic sequential quadratic programming (StoSQP) algorithm that utilizes a differentiable exact augmented Lagrangian as the merit function. The algorithm adaptively selects the penalty parameters of the augmented Lagrangian and performs a stochastic line search to decide the stepsize. The global convergence is established: for any initialization, the KKT residuals converge to zero almost surely. Our algorithm and analysis further develop the prior work of Na et al., (2022). Specifically, we allow nonlinear inequality constraints without requiring the strict complementary condition; refine some of the designs in Na et al., (2022) such as the feasibility error condition and the monotonically increasing sample size; strengthen the global convergence guarantee; and improve the sample complexity on the objective Hessian. We demonstrate the performance of the designed algorithm on a subset of nonlinear problems collected in CUTEst test set and on constrained logistic regression problems.

翻译：我们研究随机目标函数与确定性等式及不等式约束的非线性优化问题，此类问题广泛存在于金融、制造、电力系统及近期深度神经网络等应用中。提出一种基于积极集策略的随机序列二次规划（StoSQP）算法，采用可微精确增广拉格朗日函数作为价值函数。该算法自适应选取增广拉格朗日惩罚参数，并通过随机线搜索确定步长。建立了全局收敛性：对于任意初始点，KKT残差几乎必然收敛至零。本文算法与分析框架在Na等人(2022)前期工作基础上取得以下进展：允许非线性不等式约束且无需严格互补条件；优化了Na等人(2022)的若干设计，包括可行性误差条件与单调递增样本量策略；增强了全局收敛性保证；改进了目标函数Hessian矩阵的样本复杂度。我们在CUTEst测试集选定的非线性问题子集以及带约束逻辑回归问题上验证了所设计算法的性能。