In prototype-based federated learning, the exchange of model parameters between clients and the master server is replaced by transmission of prototypes or quantized versions of the data samples to the aggregation server. A fully decentralized deployment of prototype- based learning, without a central agregartor of prototypes, is more robust upon network failures and reacts faster to changes in the statistical distribution of the data, suggesting potential advantages and quick adaptation in dynamic learning tasks, e.g., when the data sources are IoT devices or when data is non-iid. In this paper, we consider the problem of designing a communication-efficient decentralized learning system based on prototypes. We address the challenge of prototype redundancy by leveraging on a twofold data compression technique, i.e., sending only update messages if the prototypes are informationtheoretically useful (via the Jensen-Shannon distance), and using clustering on the prototypes to compress the update messages used in the gossip protocol. We also use parallel instead of sequential gossiping, and present an analysis of its age-of-information (AoI). Our experimental results show that, with these improvements, the communications load can be substantially reduced without decreasing the convergence rate of the learning algorithm.

翻译：在基于原型的联邦学习中，客户端与主服务器之间的模型参数交换被替换为向聚合服务器传输数据样本的原型或其量化版本。基于原型学习的完全去中心化部署（无需中心化的原型聚合器）对网络故障具有更强的鲁棒性，并能更快地响应数据统计分布的变化，这表明其在动态学习任务中具有潜在优势与快速适应能力，例如当数据源为物联网设备或数据呈非独立同分布时。本文研究了基于原型的通信高效去中心化学习系统的设计问题。我们通过双重数据压缩技术应对原型冗余的挑战：仅当原型在信息论层面具有效用时（通过Jensen-Shannon距离判断）发送更新消息，并对原型进行聚类以压缩用于八卦协议的更新消息。此外，我们采用并行而非顺序的八卦传播机制，并对其信息年龄（AoI）进行了分析。实验结果表明，通过这些改进，可在不降低学习算法收敛速度的前提下显著减少通信负载。