We analyze the complexity of single-loop quadratic penalty and augmented Lagrangian algorithms for solving nonconvex optimization problems with functional equality constraints. We consider three cases, in all of which the objective is stochastic and smooth, that is, an expectation over an unknown distribution that is accessed by sampling. The nature of the equality constraints differs among the three cases: deterministic and linear in the first case, deterministic, smooth and nonlinear in the second case, and stochastic, smooth and nonlinear in the third case. Variance reduction techniques are used to improve the complexity. To find a point that satisfies $\varepsilon$-approximate first-order conditions, we require $\widetilde{O}(\varepsilon^{-3})$ complexity in the first case, $\widetilde{O}(\varepsilon^{-4})$ in the second case, and $\widetilde{O}(\varepsilon^{-5})$ in the third case. For the first and third cases, they are the first algorithms of "single loop" type (that also use $O(1)$ samples at each iteration) that still achieve the best-known complexity guarantees.

翻译：我们分析了单环二次罚函数法与增广拉格朗日算法在求解具有函数等式约束的非凸优化问题时的复杂度。我们考虑三种情形，所有情形中目标函数均为随机且光滑的，即通过采样访问未知分布下的期望值。三种情形的等式约束性质各异：第一种情形为确定性线性约束，第二种情形为确定性光滑非线性约束，第三种情形为随机光滑非线性约束。采用方差缩减技术以提升复杂度性能。为找到满足$\varepsilon$近似一阶条件的点，我们在第一种情形中需要的复杂度为$\widetilde{O}(\varepsilon^{-3})$，第二种情形为$\widetilde{O}(\varepsilon^{-4})$，第三种情形为$\widetilde{O}(\varepsilon^{-5})$。对于第一和第三种情形，这是首批"单环"类算法（同时每次迭代仅需$O(1)$个样本）且仍能达到已知最优复杂度保证。