In this paper, we investigate Bayesian model compression in federated learning (FL) to construct sparse models that can achieve both communication and computation efficiencies. We propose a decentralized Turbo variational Bayesian inference (D-Turbo-VBI) FL framework where we firstly propose a hierarchical sparse prior to promote a clustered sparse structure in the weight matrix. Then, by carefully integrating message passing and VBI with a decentralized turbo framework, we propose the D-Turbo-VBI algorithm which can (i) reduce both upstream and downstream communication overhead during federated training, and (ii) reduce the computational complexity during local inference. Additionally, we establish the convergence property for thr proposed D-Turbo-VBI algorithm. Simulation results show the significant gain of our proposed algorithm over the baselines in reducing communication overhead during federated training and computational complexity of final model.

翻译：本文研究了联邦学习（FL）中的贝叶斯模型压缩，以构建既能实现通信效率又能实现计算效率的稀疏模型。我们提出了一种去中心化Turbo变分贝叶斯推理（D-Turbo-VBI）联邦学习框架。在该框架中，我们首先引入一种分层稀疏先验，以促进权重矩阵中聚类稀疏结构的形成。随后，通过将消息传递和变分贝叶斯推理（VBI）与去中心化Turbo框架巧妙整合，我们提出了D-Turbo-VBI算法。该算法能够：（i）在联邦训练过程中降低上下行通信开销；（ii）在本地推理阶段降低计算复杂度。此外，我们还建立了所提D-Turbo-VBI算法的收敛性性质。仿真结果表明，与基线方法相比，所提算法在减少联邦训练阶段的通信开销和最终模型的计算复杂度方面具有显著优势。