This paper studies the stochastic optimization for decentralized nonconvex-strongly-concave minimax problem. We propose a simple and efficient algorithm, called Decentralized Recursive-gradient descEnt Ascent Method (\texttt{DREAM}), which achieves the best-known theoretical guarantee for finding the $\epsilon$-stationary point of the primal function. For the online setting, the proposed method requires $\mathcal{O}(\kappa^3\epsilon^{-3})$ stochastic first-order oracle (SFO) calls and $\mathcal{O}\big(\kappa^2\epsilon^{-2}/\sqrt{1-\lambda_2(W)}\,\big)$ communication rounds to find an $\epsilon$-stationary point, where $\kappa$ is the condition number and $\lambda_2(W)$ is the second-largest eigenvalue of the gossip matrix~$W$. For the offline setting with totally $N$ component functions, the proposed method requires $\mathcal{O}\big(\kappa^2 \sqrt{N} \epsilon^{-2}\big)$ SFO calls and the same communication complexity as the online setting.
翻译:本文研究了去中心化非凸-强凹极小极大问题的随机优化。我们提出了一种简单高效的算法,名为去中心化递归梯度上升法(\texttt{DREAM}),该算法在寻找原始函数的$\epsilon$-驻点时达到了目前最佳的理论保证。对于在线设置,所提方法需要$\mathcal{O}(\kappa^3\epsilon^{-3})$次随机一阶预言机(SFO)调用和$\mathcal{O}\big(\kappa^2\epsilon^{-2}/\sqrt{1-\lambda_2(W)}\,\big)$轮通信来找到$\epsilon$-驻点,其中$\kappa$为条件数,$\lambda_2(W)$为八卦矩阵$W$的第二大特征值。对于包含总共$N$个分量函数的离线设置,所提方法需要$\mathcal{O}\big(\kappa^2 \sqrt{N} \epsilon^{-2}\big)$次SFO调用,且通信复杂度与在线设置相同。