Distributed optimization methods with random communication skips are gaining increasing attention due to their proven benefits in accelerating communication complexity. Nevertheless, existing research mainly focuses on centralized communication protocols for strongly convex deterministic settings. In this work, we provide a decentralized optimization method called RandCom, which incorporates probabilistic local updates. We analyze the performance of RandCom in stochastic non-convex, convex, and strongly convex settings and demonstrate its ability to asymptotically reduce communication overhead by the probability of communication. Additionally, we prove that RandCom achieves linear speedup as the number of nodes increases. In stochastic strongly convex settings, we further prove that RandCom can achieve linear speedup with network-independent stepsizes. Moreover, we apply RandCom to federated learning and provide positive results concerning the potential for achieving linear speedup and the suitability of the probabilistic local update approach for non-convex settings.

翻译：利用随机通信跳过的分布式优化方法因其在加速通信复杂性方面的显著优势而受到日益关注。然而，现有研究主要集中在强凸确定性设置下的集中式通信协议。本文提出了一种名为RandCom的分布式优化方法，该方法融合了概率性局部更新。我们分析了RandCom在随机非凸、凸及强凸设置下的性能，并证明其能够通过通信概率渐近降低通信开销。此外，我们证明RandCom在节点数量增加时能实现线性加速。在随机强凸设置下，我们进一步证明RandCom可使用与网络无关的步长实现线性加速。同时，我们将RandCom应用于联邦学习，并提供了关于其在非凸设置中实现线性加速的潜力以及概率性局部更新方法的适用性的积极结果。