Federated learning is an emerging distributed machine learning framework which jointly trains a global model via a large number of local devices with data privacy protections. Its performance suffers from the non-vanishing biases introduced by the local inconsistent optimal and the rugged client-drifts by the local over-fitting. In this paper, we propose a novel and practical method, FedSpeed, to alleviate the negative impacts posed by these problems. Concretely, FedSpeed applies the prox-correction term on the current local updates to efficiently reduce the biases introduced by the prox-term, a necessary regularizer to maintain the strong local consistency. Furthermore, FedSpeed merges the vanilla stochastic gradient with a perturbation computed from an extra gradient ascent step in the neighborhood, thereby alleviating the issue of local over-fitting. Our theoretical analysis indicates that the convergence rate is related to both the communication rounds $T$ and local intervals $K$ with a upper bound $\small \mathcal{O}(1/T)$ if setting a proper local interval. Moreover, we conduct extensive experiments on the real-world dataset to demonstrate the efficiency of our proposed FedSpeed, which performs significantly faster and achieves the state-of-the-art (SOTA) performance on the general FL experimental settings than several baselines. Our code is available at \url{https://github.com/woodenchild95/FL-Simulator.git}.

翻译：联邦学习是一种新兴的分布式机器学习框架，通过大量具有数据隐私保护的本地设备联合训练全局模型。其性能受限于由本地不一致最优性引入的非消失偏差，以及由本地过拟合导致的剧烈客户端漂移。本文提出一种新颖且实用的方法——FedSpeed，以缓解这些问题带来的负面影响。具体而言，FedSpeed在当前本地更新上应用代理修正项，有效减少维持强本地一致性所必需的正则化项——代理项引入的偏差。此外，FedSpeed将原始随机梯度与邻域内额外梯度上升步骤计算的扰动相融合，从而缓解本地过拟合问题。理论分析表明，若设置合理的本地间隔，收敛速率与通信轮次 $T$ 及本地间隔 $K$ 均相关，并存在上界 $\small \mathcal{O}(1/T)$。同时，我们在真实数据集上开展大量实验，证明所提FedSpeed的高效性：在通用联邦学习实验设置下，其运行速度显著更快，且泛化准确率达到最先进的（SOTA）性能。代码开源地址为 \url{https://github.com/woodenchild95/FL-Simulator.git}。