Nonconvex-nonconcave minimax optimization has received intense attention over the last decade due to its broad applications in machine learning. Unfortunately, most existing algorithms cannot be guaranteed to converge globally and even suffer from limit cycles. To address this issue, we propose a novel single-loop algorithm called doubly smoothed gradient descent ascent method (DSGDA), which naturally balances the primal and dual updates. The proposed DSGDA can get rid of limit cycles in various challenging nonconvex-nonconcave examples in the literature, including Forsaken, Bilinearly-coupled minimax, Sixth-order polynomial, and PolarGame. We further show that under an one-sided Kurdyka-\L{}ojasiewicz condition with exponent $\theta\in(0,1)$ (resp. convex primal/concave dual function), DSGDA can find a game-stationary point with an iteration complexity of $\mathcal{O}(\epsilon^{-2\max\{2\theta,1\}})$ (resp. $\mathcal{O}(\epsilon^{-4})$). These match the best results for single-loop algorithms that solve nonconvex-concave or convex-nonconcave minimax problems, or problems satisfying the rather restrictive one-sided Polyak-\L{}ojasiewicz condition. Our work demonstrates, for the first time, the possibility of having a simple and unified single-loop algorithm for solving nonconvex-nonconcave, nonconvex-concave, and convex-nonconcave minimax problems.

翻译：非凸-非凹极小极大优化因其在机器学习中的广泛应用而在过去十年中备受关注。遗憾的是，大多数现有算法无法保证全局收敛，甚至会出现极限环。为了解决这一问题，我们提出一种新颖的单循环算法——双重平滑梯度下降上升法（DSGDA），该算法自然地平衡了原始更新和对偶更新。所提出的DSGDA能够消除文献中多种具有挑战性的非凸-非凹示例（包括Forsaken、双线性耦合极小极大、六次多项式及极地博弈）中的极限环。我们进一步证明，在指数为$\theta\in(0,1)$的单侧Kurdyka-Łojasiewicz条件（或原始凸/对偶凹函数）下，DSGDA可达到博弈稳定点，迭代复杂度为$\mathcal{O}(\epsilon^{-2\max\{2\theta,1\}})$（或$\mathcal{O}(\epsilon^{-4})$）。这些结果与解决非凸-凹或凸-非凹极小极大问题（或满足更严格单侧Polyak-Łojasiewicz条件的问题）的单循环算法的最优结果相匹配。我们的工作首次展示了采用简单且统一的单循环算法求解非凸-非凹、非凸-凹以及凸-非凹极小极大问题的可行性。