We give the first polynomial time algorithms for escaping from high-dimensional saddle points under a moderate number of constraints. Given gradient access to a smooth function $f \colon \mathbb R^d \to \mathbb R$ we show that (noisy) gradient descent methods can escape from saddle points under a logarithmic number of inequality constraints. This constitutes the first tangible progress (without reliance on NP-oracles or altering the definitions to only account for certain constraints) on the main open question of the breakthrough work of Ge et al. who showed an analogous result for unconstrained and equality-constrained problems. Our results hold for both regular and stochastic gradient descent.

翻译：我们给出了在中等数量约束下从高维鞍点中逃逸的第一个多项式时间算法。给定对光滑函数$f \colon \mathbb R^d \to \mathbb R$的梯度访问，我们证明（带噪）梯度下降方法可以在对数数量不等式约束下从鞍点中逃逸。这构成了对Ge等人突破性工作遗留主要公开问题的首个实质性进展（无需依赖NP预言机或仅针对特定约束修改定义），该工作此前给出了无约束和等式约束问题的类似结果。我们的结论对常规梯度下降和随机梯度下降均成立。