Distributed and federated learning algorithms and techniques associated primarily with minimization problems. However, with the increase of minimax optimization and variational inequality problems in machine learning, the necessity of designing efficient distributed/federated learning approaches for these problems is becoming more apparent. In this paper, we provide a unified convergence analysis of communication-efficient local training methods for distributed variational inequality problems (VIPs). Our approach is based on a general key assumption on the stochastic estimates that allows us to propose and analyze several novel local training algorithms under a single framework for solving a class of structured non-monotone VIPs. We present the first local gradient descent-accent algorithms with provable improved communication complexity for solving distributed variational inequalities on heterogeneous data. The general algorithmic framework recovers state-of-the-art algorithms and their sharp convergence guarantees when the setting is specialized to minimization or minimax optimization problems. Finally, we demonstrate the strong performance of the proposed algorithms compared to state-of-the-art methods when solving federated minimax optimization problems.

翻译：分布式和联邦学习算法与技术主要与最小化问题相关联。然而，随着机器学习中极小极大优化和变分不等式问题的增多，为这些问题设计高效分布式/联邦学习方法的需求日益凸显。本文为分布式变分不等式问题（VIPs）的通信高效局部训练方法提供了统一的收敛性分析。我们的方法基于对随机估计的一个通用关键假设，这使得我们能够在一个统一框架下提出并分析多种新型局部训练算法，用于求解一类结构化的非单调VIPs。我们首次提出了具有可证明改进通信复杂度的局部梯度下降-上升算法，用于求解异构数据上的分布式变分不等式。当设置特化为最小化或极小极大优化问题时，该通用算法框架能恢复最先进算法及其精确收敛性保证。最后，我们在求解联邦极小极大优化问题时，验证了所提算法相较于最先进方法的优越性能。