Local stochastic gradient descent (SGD) is a fundamental approach in achieving communication efficiency in Federated Learning (FL) by allowing individual workers to perform local updates. However, the presence of heterogeneous data distributions across working nodes causes each worker to update its local model towards a local optimum, leading to the phenomenon known as ``client-drift" and resulting in slowed convergence. To address this issue, previous works have explored methods that either introduce communication overhead or suffer from unsteady performance. In this work, we introduce a novel metric called ``degree of divergence," quantifying the angle between the local gradient and the global reference direction. Leveraging this metric, we propose the divergence-based adaptive aggregation (DRAG) algorithm, which dynamically ``drags" the received local updates toward the reference direction in each round without requiring extra communication overhead. Furthermore, we establish a rigorous convergence analysis for DRAG, proving its ability to achieve a sublinear convergence rate. Compelling experimental results are presented to illustrate DRAG's superior performance compared to state-of-the-art algorithms in effectively managing the client-drift phenomenon. Additionally, DRAG exhibits remarkable resilience against certain Byzantine attacks. By securely sharing a small sample of the client's data with the FL server, DRAG effectively counters these attacks, as demonstrated through comprehensive experiments.

翻译：局部随机梯度下降（SGD）是实现联邦学习（FL）通信效率的基础方法，它允许各工作节点执行本地更新。然而，工作节点间异构数据分布的存在导致每个工作节点将其本地模型更新至局部最优，引发“客户端漂移”现象，从而减缓收敛速度。为解决此问题，先前研究探索了要么引入通信开销、要么性能不稳定的方法。本文提出一种名为“散度”的新度量，用于量化局部梯度与全局参考方向之间的夹角。基于该度量，我们提出基于散度的自适应聚合（DRAG）算法，该算法在每轮迭代中动态地将接收到的局部更新“拖拽”至参考方向，且无需额外通信开销。此外，我们为DRAG建立了严格的收敛性分析，证明其能够实现次线性收敛速率。通过令人信服的实验结果，展示了DRAG在有效管理客户端漂移现象方面相较于现有最优算法的卓越性能。同时，DRAG对特定拜占庭攻击表现出显著的鲁棒性。通过安全地向FL服务器共享客户端数据的少量样本，DRAG可有效抵御此类攻击——综合实验结果已充分证实这一点。