With the promise of federated learning (FL) to allow for geographically-distributed and highly personalized services, the efficient exchange of model updates between clients and servers becomes crucial. FL, though decentralized, often faces communication bottlenecks, especially in resource-constrained scenarios. Existing data compression techniques like gradient sparsification, quantization, and pruning offer some solutions, but may compromise model performance or necessitate expensive retraining. In this paper, we introduce FedSZ, a specialized lossy-compression algorithm designed to minimize the size of client model updates in FL. FedSZ incorporates a comprehensive compression pipeline featuring data partitioning, lossy and lossless compression of model parameters and metadata, and serialization. We evaluate FedSZ using a suite of error-bounded lossy compressors, ultimately finding SZ2 to be the most effective across various model architectures and datasets including AlexNet, MobileNetV2, ResNet50, CIFAR-10, Caltech101, and Fashion-MNIST. Our study reveals that a relative error bound 1E-2 achieves an optimal tradeoff, compressing model states between 5.55-12.61x while maintaining inference accuracy within <0.5% of uncompressed results. Additionally, the runtime overhead of FedSZ is <4.7% or between of the wall-clock communication-round time, a worthwhile trade-off for reducing network transfer times by an order of magnitude for networks bandwidths <500Mbps. Intriguingly, we also find that the error introduced by FedSZ could potentially serve as a source of differentially private noise, opening up new avenues for privacy-preserving FL.

翻译：联邦学习（FL）因其能够支持地理分布式及高度个性化服务而备受关注，在此背景下，客户端与服务器间模型更新的高效交换变得至关重要。尽管联邦学习采用去中心化架构，但其常面临通信瓶颈，尤其在资源受限场景中尤为突出。现有的梯度稀疏化、量化和剪枝等数据压缩技术虽能提供部分解决方案，但可能牺牲模型性能或需要昂贵的重训练成本。本文提出FedSZ——一种专为最小化联邦学习中客户端模型更新规模而设计的有损压缩算法。FedSZ集成了一套完整的压缩流水线，包含数据分区、模型参数与元数据的无损/有损混合压缩及序列化操作。我们通过一系列误差有界有损压缩器对FedSZ进行评估，最终发现SZ2在多种模型架构与数据集（包括AlexNet、MobileNetV2、ResNet50、CIFAR-10、Caltech101和Fashion-MNIST）中表现最优。研究表明，相对误差界1E-2可实现最优权衡，在将模型状态压缩5.55-12.61倍的同时，推理精度损失控制在未压缩结果的0.5%以内。此外，FedSZ的运行时间开销低于通信轮次壁钟时间的4.7%，这一代价可换取在网络带宽低于500Mbps时降低一个数量级的网络传输时间。值得关注的是，我们意外发现FedSZ引入的误差可能作为差分隐私噪声源，为隐私保护联邦学习开辟了新途径。