Federated learning is a machine learning training paradigm that enables clients to jointly train models without sharing their own localized data. However, the implementation of federated learning in practice still faces numerous challenges, such as the large communication overhead due to the repetitive server-client synchronization and the lack of adaptivity by SGD-based model updates. Despite that various methods have been proposed for reducing the communication cost by gradient compression or quantization, and the federated versions of adaptive optimizers such as FedAdam are proposed to add more adaptivity, the current federated learning framework still cannot solve the aforementioned challenges all at once. In this paper, we propose a novel communication-efficient adaptive federated learning method (FedCAMS) with theoretical convergence guarantees. We show that in the nonconvex stochastic optimization setting, our proposed FedCAMS achieves the same convergence rate of $O(\frac{1}{\sqrt{TKm}})$ as its non-compressed counterparts. Extensive experiments on various benchmarks verify our theoretical analysis.

翻译：联邦学习是一种机器学习训练范式，使客户端能够在不共享本地数据的情况下联合训练模型。然而，联邦学习在实际部署中仍面临诸多挑战，例如因服务器与客户端之间反复同步导致的大量通信开销，以及基于SGD的模型更新缺乏自适应性。尽管已有多种方法通过梯度压缩或量化降低通信成本，并提出诸如FedAdam等自适应优化器的联邦版本以增强自适应性，但当前的联邦学习框架仍无法同时解决上述挑战。本文提出了一种具有理论收敛保证的新型通信高效自适应联邦学习方法（FedCAMS）。我们证明，在非凸随机优化场景下，所提出的FedCAMS能达到与未压缩方法相同的收敛速率$O(\frac{1}{\sqrt{TKm}})$。在多种基准测试上的广泛实验验证了我们的理论分析。