Federated Learning (FL) is a distributed machine learning paradigm that enables learning models from decentralized local data. While FL offers appealing properties for clients' data privacy, it imposes high communication burdens for exchanging model weights between a server and the clients. Existing approaches rely on model compression techniques, such as pruning and weight clustering to tackle this. However, transmitting the entire set of weight updates at each federated round, even in a compressed format, limits the potential for a substantial reduction in communication volume. We propose FedCode where clients transmit only codebooks, i.e., the cluster centers of updated model weight values. To ensure a smooth learning curve and proper calibration of clusters between the server and the clients, FedCode periodically transfers model weights after multiple rounds of solely communicating codebooks. This results in a significant reduction in communication volume between clients and the server in both directions, without imposing significant computational overhead on the clients or leading to major performance degradation of the models. We evaluate the effectiveness of FedCode using various publicly available datasets with ResNet-20 and MobileNet backbone model architectures. Our evaluations demonstrate a 12.2-fold data transmission reduction on average while maintaining a comparable model performance with an average accuracy loss of 1.3% compared to FedAvg. Further validation of FedCode performance under non-IID data distributions showcased an average accuracy loss of 2.0% compared to FedAvg while achieving approximately a 12.7-fold data transmission reduction.

翻译：联邦学习（FL）是一种分布式机器学习范式，能够从分散的本地数据中学习模型。尽管FL在保护客户端数据隐私方面具有吸引人的特性，但在服务器与客户端之间交换模型权重时会产生较高的通信负担。现有方法依赖模型压缩技术（如剪枝和权重聚类）来解决这一问题。然而，即使在压缩格式下，每个联邦轮次中传输完整的权重更新集仍限制了通信量的大幅降低潜力。我们提出FedCode方法，客户端仅传输码本（即更新模型权重值的聚类中心）。为确保服务器与客户端之间的学习曲线平滑且聚类校准得当，FedCode在仅传输码本的多个轮次后周期性地传输完整模型权重。这使得客户端与服务器之间的双向通信量显著降低，却不会给客户端带来显著计算开销，也不会导致模型性能大幅下降。我们使用ResNet-20和MobileNet骨干模型架构，基于多种公开数据集评估了FedCode的有效性。评估表明，与FedAvg相比，FedCode在保持可比较模型性能（平均准确率损失1.3%）的同时，数据传输量平均降低12.2倍。在非独立同分布（non-IID）数据分布下对FedCode性能的进一步验证显示，与FedAvg相比，其平均准确率损失为2.0%，同时数据传输量降低约12.7倍。