We consider statistical inference of equality-constrained stochastic nonlinear optimization problems. We develop a fully online stochastic sequential quadratic programming (StoSQP) method to solve the problems, which can be regarded as applying Newton's method to the first-order optimality conditions (i.e., the KKT conditions). Motivated by recent designs of numerical second-order methods, we allow StoSQP to adaptively select any random stepsize $\bar{\alpha}_t$, as long as $\beta_t\leq \bar{\alpha}_t \leq \beta_t+\chi_t$, for some control sequences $\beta_t$ and $\chi_t=o(\beta_t)$. To reduce the dominant computational cost of second-order methods, we additionally allow StoSQP to inexactly solve quadratic programs via efficient randomized iterative solvers that utilize sketching techniques. Notably, we do not require the approximation error to diminish as iteration proceeds. For the developed method, we show that under mild assumptions (i) computationally, it can take at most $O(1/\epsilon^4)$ iterations (same as samples) to attain $\epsilon$-stationarity; (ii) statistically, its primal-dual sequence $1/\sqrt{\beta_t}\cdot (x_t - x^\star, \lambda_t - \lambda^\star)$ converges to a mean-zero Gaussian distribution with a nontrivial covariance matrix depending on the underlying sketching distribution. Additionally, we establish the almost-sure convergence rate of the iterate $(x_t, \lambda_t)$ along with the Berry-Esseen bound; the latter quantitatively measures the convergence rate of the distribution function. We analyze a plug-in limiting covariance matrix estimator, and demonstrate the performance of the method both on benchmark nonlinear problems in CUTEst test set and on linearly/nonlinearly constrained regression problems.

翻译：我们考虑了等式约束随机非线性优化问题的统计推断问题。为此，我们开发了一种完全在线化的随机序列二次规划（StoSQP）方法，该方法可视为将牛顿法应用于一阶最优性条件（即KKT条件）。受近期数值二阶方法设计的启发，我们允许StoSQP自适应地选择任意随机步长$\bar{\alpha}_t$，只要满足$\beta_t\leq \bar{\alpha}_t \leq \beta_t+\chi_t$（其中$\beta_t$和$\chi_t=o(\beta_t)$为控制序列）。为降低二阶方法的计算成本主导问题，我们进一步允许StoSQP通过利用草图化技术的高效随机迭代求解器来非精确求解二次规划。值得注意的是，我们并不要求近似误差随迭代而减小。针对所提出的方法，我们证明在温和假设下：（i）计算方面，该方法最多需要$O(1/\epsilon^4)$次迭代（等同于样本量）即可达到$\epsilon$精度平稳点；（ii）统计方面，其原始-对偶序列$1/\sqrt{\beta_t}\cdot (x_t - x^\star, \lambda_t - \lambda^\star)$依分布收敛于均值为零的高斯分布，其协方差矩阵依赖于底层草图化分布。此外，我们建立了迭代点$(x_t, \lambda_t)$的几乎必然收敛速度以及Berry-Esseen界，后者从量化角度衡量分布函数的收敛速度。我们分析了基于插值的极限协方差矩阵估计量，并在CUTEst测试集中的基准非线性问题以及线性/非线性约束回归问题上验证了方法的有效性。