Many compression techniques have been proposed to reduce the communication overhead of Federated Learning training procedures. However, these are typically designed for compressing model updates, which are expected to decay throughout training. As a result, such methods are inapplicable to downlink (i.e., from the parameter server to clients) compression in the cross-device setting, where heterogeneous clients $\textit{may appear only once}$ during training and thus must download the model parameters. Accordingly, we propose $\textsf{DoCoFL}$ -- a new framework for downlink compression in the cross-device setting. Importantly, $\textsf{DoCoFL}$ can be seamlessly combined with many uplink compression schemes, rendering it suitable for bi-directional compression. Through extensive evaluation, we show that $\textsf{DoCoFL}$ offers significant bi-directional bandwidth reduction while achieving competitive accuracy to that of a baseline without any compression.
翻译:[翻译后的摘要] 为降低联邦学习训练过程的通信开销,已有多种压缩技术被提出。然而,这些技术通常针对模型更新的压缩设计,而模型更新在训练过程中会逐渐衰减。因此,在跨设备场景中(异构客户端在训练期间$\textit{可能仅出现一次}$并必须下载完整模型参数),此类方法无法适用于下行链路(即从参数服务器到客户端)的压缩。为此,我们提出$\textsf{DoCoFL}$——一个面向跨设备场景下行链路压缩的新框架。重要之处在于,$\textsf{DoCoFL}$可与多种上行链路压缩方案无缝结合,使其适用于双向压缩。通过大量评估,我们证明$\textsf{DoCoFL}$能在实现显著双向带宽缩减的同时,保持与无压缩基线相当的精度。