Due to the expanding scope of machine learning (ML) to the fields of sensor networking, cooperative robotics and many other multi-agent systems, distributed deployment of inference algorithms has received a lot of attention. These algorithms involve collaboratively learning unknown parameters from dispersed data collected by multiple agents. There are two competing aspects in such algorithms, namely, intra-agent computation and inter-agent communication. Traditionally, algorithms are designed to perform both synchronously. However, certain circumstances need frugal use of communication channels as they are either unreliable, time-consuming, or resource-expensive. In this paper, we propose gossip-based asynchronous communication to leverage fast computations and reduce communication overhead simultaneously. We analyze the effects of multiple (local) intra-agent computations by the active agents between successive inter-agent communications. For local computations, Bayesian sampling via unadjusted Langevin algorithm (ULA) MCMC is utilized. The communication is assumed to be over a connected graph (e.g., as in decentralized learning), however, the results can be extended to coordinated communication where there is a central server (e.g., federated learning). We theoretically quantify the convergence rates in the process. To demonstrate the efficacy of the proposed algorithm, we present simulations on a toy problem as well as on real world data sets to train ML models to perform classification tasks. We observe faster initial convergence and improved performance accuracy, especially in the low data range. We achieve on average 78% and over 90% classification accuracy respectively on the Gamma Telescope and mHealth data sets from the UCI ML repository.

翻译：随着机器学习（ML）在传感器网络、协作机器人及众多多智能体系统领域的应用范围不断扩大，分布式推理算法的部署受到广泛关注。这类算法涉及从多个智能体收集的分散数据中协同学习未知参数。此类算法存在两个相互制约的方面，即智能体内部计算与智能体间通信。传统上，算法设计采用同步执行模式。然而，在特定场景中，通信信道因不可靠、耗时或资源昂贵而需节约使用。本文提出基于闲谈（gossip）机制的异步通信方法，以兼顾快速计算与降低通信开销。我们分析了活跃智能体在连续智能体间通信之间执行多次（局部）内部计算的影响。局部计算采用通过未调整朗之万算法（ULA）MCMC实现的贝叶斯采样。假设通信发生在连通图上（如去中心化学习），但研究结果可扩展至存在中央服务器的协调通信场景（如联邦学习）。我们从理论上量化了该过程的收敛速率。为验证所提算法的有效性，我们在玩具问题及真实数据集上开展仿真实验，用于训练ML分类模型。实验观察到更快的初始收敛速度与改进的精度表现，尤其在低数据量范围内表现突出。在UCI ML知识库的Gamma Telescope与mHealth数据集上，分别实现了平均78%与超过90%的分类准确率。