Many important machine learning applications amount to solving minimax optimization problems, and in many cases there is no access to the gradient information, but only the function values. In this paper, we focus on such a gradient-free setting, and consider the nonconvex-strongly-concave minimax stochastic optimization problem. In the literature, various zeroth-order (i.e., gradient-free) minimax methods have been proposed, but none of them achieve the potentially feasible computational complexity of $\mathcal{O}(\epsilon^{-3})$ suggested by the stochastic nonconvex minimization theorem. In this paper, we adopt the variance reduction technique to design a novel zeroth-order variance reduced gradient descent ascent (ZO-VRGDA) algorithm. We show that the ZO-VRGDA algorithm achieves the best known query complexity of $\mathcal{O}(\kappa(d_1 + d_2)\epsilon^{-3})$, which outperforms all previous complexity bound by orders of magnitude, where $d_1$ and $d_2$ denote the dimensions of the optimization variables and $\kappa$ denotes the condition number. In particular, with a new analysis technique that we develop, our result does not rely on a diminishing or accuracy-dependent stepsize usually required in the existing methods. To our best knowledge, this is the first study of zeroth-order minimax optimization with variance reduction. Experimental results on the black-box distributional robust optimization problem demonstrates the advantageous performance of our new algorithm.

翻译：许多重要的机器学习应用程序都相当于解决迷你马克思优化问题, 在许多情况下, 无法获取梯度信息, 但只有函数值 。 在本文中, 我们聚焦于这种无梯度的设置, 并且考虑非convex- 强调小型摩托式优化问题 。 在文献中, 提出了各种零顺序( 即无梯度) 迷你最大优化方法 。 但是, 其中没有一种方法能够实现 $mathcal{O} ( epsilon ⁇ -3} ) 可能可行的计算复杂度, 并且无法获取 梯度信息 。 在本文中, 我们将这种零级零级差异降低梯度差异降为精度( ZO- VRGDA) 算法。 我们的 ZO- VRGDA 算法达到了最已知的查询复杂度 { O} ( kappapa( d_ 1 + d_ box) ( legality) 3} $( levelloom) $), levelop levelop levelopmental lement of levelop lementalizational levelop levelop ledge laft lection) ledge lement lemental lection a level roforizes) roforizes, legislate a lections a d daltiquest proforizes) legleglegleglegleglementaltitionaltitional roglements) lementals) 。 ex ex rogisal rogleglex roglements ( leg) rog)