Distributed and federated learning algorithms and techniques associated primarily with minimization problems. However, with the increase of minimax optimization and variational inequality problems in machine learning, the necessity of designing efficient distributed/federated learning approaches for these problems is becoming more apparent. In this paper, we provide a unified convergence analysis of communication-efficient local training methods for distributed variational inequality problems (VIPs). Our approach is based on a general key assumption on the stochastic estimates that allows us to propose and analyze several novel local training algorithms under a single framework for solving a class of structured non-monotone VIPs. We present the first local gradient descent-accent algorithms with provable improved communication complexity for solving distributed variational inequalities on heterogeneous data. The general algorithmic framework recovers state-of-the-art algorithms and their sharp convergence guarantees when the setting is specialized to minimization or minimax optimization problems. Finally, we demonstrate the strong performance of the proposed algorithms compared to state-of-the-art methods when solving federated minimax optimization problems.

翻译：分布式与联邦学习算法及相关技术主要与最小化问题相关联。然而，随着机器学习中极小极大优化和变分不等式问题的增加，为这些问题设计高效分布式/联邦学习方法的必要性日益凸显。本文针对分布式变分不等式问题，对通信高效的局部训练方法进行了统一的收敛性分析。我们的方法基于对随机估计量的一般性关键假设，该假设使我们能够在单一框架下提出并分析多种新颖的局部训练算法，用于求解一类结构化非单调变分不等式问题。我们首次提出了具有可证明改进通信复杂度的局部梯度下降-上升算法，用于求解异构数据上的分布式变分不等式问题。当设置特化为最小化或极小极大优化问题时，该通用算法框架能够恢复现有最优算法及其精确收敛性保证。最后，在求解联邦极小极大优化问题时，我们通过实验证明了所提算法相较于现有最优方法的优越性能。