Communication compression, a technique aiming to reduce the information volume to be transmitted over the air, has gained great interests in Federated Learning (FL) for the potential of alleviating its communication overhead. However, communication compression brings forth new challenges in FL due to the interplay of compression-incurred information distortion and inherent characteristics of FL such as partial participation and data heterogeneity. Despite the recent development, the performance of compressed FL approaches has not been fully exploited. The existing approaches either cannot accommodate arbitrary data heterogeneity or partial participation, or require stringent conditions on compression. In this paper, we revisit the seminal stochastic controlled averaging method by proposing an equivalent but more efficient/simplified formulation with halved uplink communication costs. Building upon this implementation, we propose two compressed FL algorithms, SCALLION and SCAFCOM, to support unbiased and biased compression, respectively. Both the proposed methods outperform the existing compressed FL methods in terms of communication and computation complexities. Moreover, SCALLION and SCAFCOM accommodates arbitrary data heterogeneity and do not make any additional assumptions on compression errors. Experiments show that SCALLION and SCAFCOM can match the performance of corresponding full-precision FL approaches with substantially reduced uplink communication, and outperform recent compressed FL methods under the same communication budget.

翻译：通信压缩是一种旨在减少空中传输信息量的技术，因其缓解联邦学习（FL）通信开销的潜力而受到广泛关注。然而，由于压缩引起的信息失真与FL固有特性（如部分参与和数据异构性）的相互作用，通信压缩给FL带来了新的挑战。尽管近期有所发展，但压缩FL方法的性能尚未得到充分利用。现有方法要么无法适应任意数据异构性或部分参与场景，要么对压缩施加了严格条件。本文通过提出一种等效但更高效/简洁的实现方案（将上行通信成本减半），重新审视了经典的随机控制平均法。基于该实现，我们提出了两种压缩FL算法——SCALLION和SCAFCOM，分别支持无偏压缩和有偏压缩。这两种方法在通信和计算复杂度上均优于现有压缩FL方法。此外，SCALLION和SCAFCOM能适应任意数据异构性，且不对压缩误差做额外假设。实验表明，SCALLION和SCAFCOM能以显著减少的上行通信量达到对应全精度FL方法的性能，并在相同通信预算下优于近期提出的压缩FL方法。