As an emerging federated learning paradigm, federated distillation enables communication-efficient model training by transmitting only small-scale knowledge during the learning process. To further improve the communication efficiency of federated distillation, we propose a novel technique, ALU, which accumulates multiple rounds of local updates before transferring the knowledge to the central server. ALU drastically decreases the frequency of communication in federated distillation, thereby significantly reducing the communication overhead during the training process. Empirical experiments demonstrate the substantial effect of ALU in improving the communication efficiency of federated distillation.

翻译：作为一种新兴的联邦学习范式，联邦蒸馏通过在训练过程中仅传输小规模知识，实现了通信高效的模型训练。为进一步提升联邦蒸馏的通信效率，我们提出了一种名为ALU的创新技术。该技术在将知识传输至中央服务器之前，累积多轮本地更新。ALU大幅降低了联邦蒸馏中的通信频率，从而显著减少了训练过程中的通信开销。实证实验表明，ALU在提升联邦蒸馏通信效率方面具有显著效果。