Due to the expanding scope of machine learning (ML) to the fields of sensor networking, cooperative robotics and many other multi-agent systems, distributed deployment of inference algorithms has received a lot of attention. These algorithms involve collaboratively learning unknown parameters from dispersed data collected by multiple agents. There are two competing aspects in such algorithms, namely, intra-agent computation and inter-agent communication. Traditionally, algorithms are designed to perform both synchronously. However, certain circumstances need frugal use of communication channels as they are either unreliable, time-consuming, or resource-expensive. In this paper, we propose gossip-based asynchronous communication to leverage fast computations and reduce communication overhead simultaneously. We analyze the effects of multiple (local) intra-agent computations by the active agents between successive inter-agent communications. For local computations, Bayesian sampling via unadjusted Langevin algorithm (ULA) MCMC is utilized. The communication is assumed to be over a connected graph (e.g., as in decentralized learning), however, the results can be extended to coordinated communication where there is a central server (e.g., federated learning). We theoretically quantify the convergence rates in the process. To demonstrate the efficacy of the proposed algorithm, we present simulations on a toy problem as well as on real world data sets to train ML models to perform classification tasks. We observe faster initial convergence and improved performance accuracy, especially in the low data range. We achieve on average 78% and over 90% classification accuracy respectively on the Gamma Telescope and mHealth data sets from the UCI ML repository.

翻译：随着机器学习在传感器网络、协作机器人及众多多智能体系统领域的应用范围不断扩大，分布式推理算法的部署受到广泛关注。这类算法涉及从多个智能体采集的分散数据中协同学习未知参数。此类算法存在两个相互制约的方面，即智能体内部计算与智能体间通信。传统算法通常采用同步方式执行这两类操作。然而在某些场景下，由于通信信道不可靠、耗时或资源昂贵，需要节约使用。本文提出基于八卦机制的异步通信方案，旨在同时实现快速计算与降低通信开销。我们分析了活跃智能体在连续两次通信间隔内执行多次（局部）内部计算的影响。局部计算采用未调整朗之万算法马尔可夫链蒙特卡洛方法进行贝叶斯采样。通信假设在连通图结构上进行（如去中心化学习场景），但结果可扩展至存在中央服务器的协调通信模式（如联邦学习）。我们从理论上量化了该过程的收敛速率。为验证所提算法的有效性，分别在玩具问题及真实数据集上开展仿真实验，训练机器学习模型执行分类任务。实验观察到更快的初始收敛速度与更高的精度，尤其在低数据量场景下表现显著。在UCI机器学习库的伽马望远镜数据集和移动健康数据集上，我们分别实现了平均78%和超过90%的分类准确率。