We study the complexity of producing $(\delta,\epsilon)$-stationary points of Lipschitz objectives which are possibly neither smooth nor convex, using only noisy function evaluations. Recent works proposed several stochastic zero-order algorithms that solve this task, all of which suffer from a dimension-dependence of $\Omega(d^{3/2})$ where $d$ is the dimension of the problem, which was conjectured to be optimal. We refute this conjecture by providing a faster algorithm that has complexity $O(d\delta^{-1}\epsilon^{-3})$, which is optimal (up to numerical constants) with respect to $d$ and also optimal with respect to the accuracy parameters $\delta,\epsilon$, thus solving an open question due to Lin et al. (NeurIPS'22). Moreover, the convergence rate achieved by our algorithm is also optimal for smooth objectives, proving that in the nonconvex stochastic zero-order setting, nonsmooth optimization is as easy as smooth optimization. We provide algorithms that achieve the aforementioned convergence rate in expectation as well as with high probability. Our analysis is based on a simple yet powerful geometric lemma regarding the Goldstein-subdifferential set, which allows utilizing recent advancements in first-order nonsmooth nonconvex optimization.

翻译：我们研究了在仅使用含噪函数评估的条件下，生成Lipschitz目标函数（可能既不光滑也不凸）的$(\delta,\epsilon)$-驻点的复杂度。近期工作提出了若干解决该任务的随机零阶算法，这些算法均受到$\Omega(d^{3/2})$的维度依赖性限制（其中$d$为问题维度），此前被认为是最优的。我们推翻了这一猜想，提出了一种更快的算法，其复杂度为$O(d\delta^{-1}\epsilon^{-3})$，在$d$以及精度参数$\delta,\epsilon$方面均达到最优（数值常数意义下），从而解决了Lin等人(NeurIPS'22)提出的公开问题。此外，我们的算法在光滑目标函数上也实现了最优收敛率，证明在非凸随机零阶设定中，非光滑优化与光滑优化同样简单。我们提供的算法在期望意义下以及高概率意义上均能达到上述收敛率。我们的分析基于一个关于Goldstein-次微分集的简洁而强大的几何引理，该引理使得我们能够利用一阶非光滑非凸优化的最新进展。