In Distributed optimization and Learning, and even more in the modern framework of federated learning, communication, which is slow and costly, is critical. We introduce LoCoDL, a communication-efficient algorithm that leverages the two popular and effective techniques of Local training, which reduces the communication frequency, and Compression, in which short bitstreams are sent instead of full-dimensional vectors of floats. LoCoDL works with a large class of unbiased compressors that includes widely-used sparsification and quantization methods. LoCoDL provably benefits from local training and compression and enjoys a doubly-accelerated communication complexity, with respect to the condition number of the functions and the model dimension, in the general heterogenous regime with strongly convex functions. This is confirmed in practice, with LoCoDL outperforming existing algorithms.

翻译：在分布式优化和学习中，尤其是在现代联邦学习框架下，通信过程缓慢且成本高昂，已成为关键瓶颈。我们提出LoCoDL——一种通信高效的算法，它融合了两种广泛使用且有效的技术：局部训练（降低通信频率）与压缩（发送短比特流而非全维度浮点向量）。该算法适用于包含广泛使用的稀疏化与量化方法在内的无偏压缩器大类。LoCoDL可证明地受益于局部训练与压缩，并在函数条件数与模型维度方面双重加速通信复杂度，适用于强凸函数的一般异构场景。这一优势在实践中得到验证，LoCoDL的性能优于现有算法。