In federated learning, a large number of users are involved in a global learning task, in a collaborative way. They alternate local computations and two-way communication with a distant orchestrating server. Communication, which can be slow and costly, is the main bottleneck in this setting. To reduce the communication load and therefore accelerate distributed gradient descent, two strategies are popular: 1) communicate less frequently; that is, perform several iterations of local computations between the communication rounds; and 2) communicate compressed information instead of full-dimensional vectors. We propose the first algorithm for distributed optimization and federated learning, which harnesses these two strategies jointly and converges linearly to an exact solution in the strongly convex setting, with a doubly accelerated rate: our algorithm benefits from the two acceleration mechanisms provided by local training and compression, namely a better dependency on the condition number of the functions and on the dimension of the model, respectively.

翻译：在联邦学习中，大量用户以协作方式参与全局学习任务。用户交替执行本地计算，并与远程协调服务器进行双向通信。通信过程可能缓慢且成本高昂，是该场景下的主要瓶颈。为降低通信负载从而加速分布式梯度下降，当前流行两种策略：1）降低通信频率，即在通信轮次之间执行多次本地计算迭代；2）传输压缩信息而非完整维度向量。我们提出首个联合利用这两种策略的分布式优化与联邦学习算法，该算法在强凸条件下能以双重加速速率线性收敛至精确解：我们的算法分别受益于本地训练与压缩提供的两种加速机制，即分别降低对函数条件数和模型维度的依赖。