Nonconvex-nonconcave minimax optimization has received intense attention over the last decade due to its broad applications in machine learning. Unfortunately, most existing algorithms cannot be guaranteed to converge globally and even suffer from limit cycles. To address this issue, we propose a novel single-loop algorithm called doubly smoothed gradient descent ascent method (DSGDA), which naturally balances the primal and dual updates. The proposed DSGDA can get rid of limit cycles in various challenging nonconvex-nonconcave examples in the literature, including Forsaken, Bilinearly-coupled minimax, Sixth-order polynomial, and PolarGame. We further show that under an one-sided Kurdyka-\L{}ojasiewicz condition with exponent $\theta\in(0,1)$ (resp. convex primal/concave dual function), DSGDA can find a game-stationary point with an iteration complexity of $\mathcal{O}(\epsilon^{-2\max\{2\theta,1\}})$ (resp. $\mathcal{O}(\epsilon^{-4})$). These match the best results for single-loop algorithms that solve nonconvex-concave or convex-nonconcave minimax problems, or problems satisfying the rather restrictive one-sided Polyak-\L{}ojasiewicz condition. Our work demonstrates, for the first time, the possibility of having a simple and unified single-loop algorithm for solving nonconvex-nonconcave, nonconvex-concave, and convex-nonconcave minimax problems.

翻译：非凸-非凹极小极大优化因其在机器学习中的广泛应用，过去十年间受到高度关注。然而，现有大多数算法无法保证全局收敛，甚至可能出现极限环。为解决此问题，我们提出一种新型单循环算法——双平滑梯度下降上升法（DSGDA），该算法自然地平衡了原始更新与对偶更新。所提出的DSGDA能够消除文献中多种具有挑战性的非凸-非凹示例中的极限环，包括Forsaken、双线性耦合极小极大、六阶多项式及PolarGame。我们进一步证明，在指数为$\theta\in(0,1)$的单侧Kurdyka-Łojasiewicz条件（或凸原始/凹对偶函数）下，DSGDA能以$\mathcal{O}(\epsilon^{-2\max\{2\theta,1\}})$（或$\mathcal{O}(\epsilon^{-4})$）的迭代复杂度找到博弈驻点。这些结果匹配了求解非凸-凹或凸-非凹极小极大问题，或满足较强单侧Polyak-Łojasiewicz条件问题的最优单循环算法结果。我们的工作首次展示了使用简单统一的单循环算法求解非凸-非凹、非凸-凹及凸-非凹极小极大问题的可能性。