Compression schemes have been extensively used in Federated Learning (FL) to reduce the communication cost of distributed learning. While most approaches rely on a bounded variance assumption of the noise produced by the compressor, this paper investigates the use of compression and aggregation schemes that produce a specific error distribution, e.g., Gaussian or Laplace, on the aggregated data. We present and analyze different aggregation schemes based on layered quantizers achieving exact error distribution. We provide different methods to leverage the proposed compression schemes to obtain compression-for-free in differential privacy applications. Our general compression methods can recover and improve standard FL schemes with Gaussian perturbations such as Langevin dynamics and randomized smoothing.

翻译：压缩方案已被广泛应用于联邦学习中，以降低分布式学习的通信成本。尽管大多数方法依赖于压缩器产生噪声的有界方差假设，但本文研究了能够对聚合数据产生特定误差分布（例如高斯分布或拉普拉斯分布）的压缩与聚合方案。我们提出并分析了基于分层量化器实现精确误差分布的不同聚合方案，提供了多种方法来利用所提出的压缩方案，在差分隐私应用中实现"免费压缩"。我们的通用压缩方法能够恢复并改进具有高斯扰动的标准联邦学习方案，例如朗之万动力学和随机平滑。