We consider distributed stochastic variational inequalities (VIs) on unbounded domain with the problem data being heterogeneous (non-IID) and distributed across many devices. We make very general assumption on the computational network that, in particular, covers the settings of fully decentralized calculations with time-varying networks and centralized topologies commonly used in Federated Learning. Moreover, multiple local updates on the workers can be made for reducing the communication frequency between workers. We extend stochastic extragradient method to this very general setting and theoretically analyze its convergence rate in the strongly monotone, monotone, and non-monotone setting when an Minty solution exists. The provided rates have explicit dependence on\ network characteristics and how it varies with time, data heterogeneity, variance, number of devices, and other standard parameters. As a special case, our method and analysis apply to distributed stochastic saddle-point problems (SPP), e.g., to training Deep Generative Adversarial Networks (GANs) for which the decentralized training has been reported to be extremely challenging. In experiments for decentralized training of GANs we demonstrate the effectiveness of our proposed approach.

翻译：我们认为,在非封闭域上分布的随机差异性不平等(VIs)是分布式的,问题数据是混杂的(非二维),并且分布在许多装置上。我们对计算网络作了非常笼统的假设,特别是覆盖了与联邦学习联合会常用的时间变化网络和集中型地形完全分散的计算环境;此外,可以对工人进行多次当地更新,以减少工人之间的沟通频率。我们将随机异异变方法推广到这一非常笼统的设置上,并从理论上分析在极强的单体内、单体内和非单体内设置时的趋同率。所提供的比率明显依赖\网络特性及其与时间、数据多样性、差异、装置数目和其他标准参数的不同。作为一个特殊的例子,我们的方法和分析适用于传播的随机性马垫点问题(SPP),例如,培训深基因对面网络(GANs),因为据报告,分散式的培训对后者具有极大的挑战。在对GANs进行分散式方法的实验中,我们展示了我们所提议的方法和分析的有效性。