Federated Learning (FL) is a promising technique for the collaborative training of deep neural networks across multiple devices while preserving data privacy. Despite its potential benefits, FL is hindered by excessive communication costs due to repeated server-client communication during training. To address this challenge, model compression techniques, such as sparsification and weight clustering are applied, which often require modifying the underlying model aggregation schemes or involve cumbersome hyperparameter tuning, with the latter not only adjusts the model's compression rate but also limits model's potential for continuous improvement over growing data. In this paper, we propose FedCompress, a novel approach that combines dynamic weight clustering and server-side knowledge distillation to reduce communication costs while learning highly generalizable models. Through a comprehensive evaluation on diverse public datasets, we demonstrate the efficacy of our approach compared to baselines in terms of communication costs and inference speed. We will make our implementation public upon acceptance.

翻译：联邦学习（FL）是一种极具前景的技术，可在保护数据隐私的前提下跨多个设备协同训练深度神经网络。尽管具有潜在优势，但训练过程中的重复服务器-客户端通信导致通信成本过高，严重制约了联邦学习的发展。为解决这一挑战，现有研究采用稀疏化、权值聚类等模型压缩技术，但这些方法往往需要修改底层模型聚合方案，或涉及繁琐的超参数调优——后者不仅调整模型压缩率，更限制了模型在数据持续增长中的持续优化潜力。本文提出FedCompress创新方法，通过融合动态权值聚类与服务器端知识蒸馏技术，在降低通信成本的同时保持模型强泛化能力。基于多样化的公开数据集进行的全面评估表明，相较于基线方法，本方法在通信效率和推理速度方面具有显著优势。代码将在论文接收后公开。