Federated learning can train models without directly providing local data to the server. However, the frequent updating of the local model brings the problem of large communication overhead. Recently, scholars have achieved the communication efficiency of federated learning mainly by model compression. But they ignore two problems: 1) network state of each client changes dynamically; 2) network state among clients is not the same. The clients with poor bandwidth update local model slowly, which leads to low efficiency. To address this challenge, we propose a communication-efficient federated learning algorithm with adaptive compression under dynamic bandwidth (called AdapComFL). Concretely, each client performs bandwidth awareness and bandwidth prediction. Then, each client adaptively compresses its local model via the improved sketch mechanism based on his predicted bandwidth. Further, the server aggregates sketched models with different sizes received. To verify the effectiveness of the proposed method, the experiments are based on real bandwidth data which are collected from the network topology we build, and benchmark datasets which are obtained from open repositories. We show the performance of AdapComFL algorithm, and compare it with existing algorithms. The experimental results show that our AdapComFL achieves more efficient communication as well as competitive accuracy compared to existing algorithms.
翻译:联邦学习可以在不直接向服务器提供本地数据的情况下训练模型。然而,本地模型的频繁更新带来了通信开销大的问题。近期,学者主要通过模型压缩实现了联邦学习的通信效率,但忽略了两个问题:1) 各客户端的网络状态动态变化;2) 各客户端之间的网络状态存在差异。带宽较差的客户端本地模型更新缓慢,导致效率低下。针对这一挑战,我们提出了一种动态带宽下自适应压缩的高效通信联邦学习算法(称为AdapComFL)。具体地,每个客户端进行带宽感知和带宽预测。然后,每个客户端基于其预测的带宽,通过改进的草图机制自适应压缩其本地模型。此外,服务器聚合接收到的不同大小的草图模型。为了验证所提方法的有效性,实验基于从我们构建的网络拓扑中收集的真实带宽数据,以及从公开数据集中获取的基准数据集。我们展示了AdapComFL算法的性能,并与现有算法进行了比较。实验结果表明,与现有算法相比,我们的AdapComFL实现了更高效的通信以及有竞争力的精度。