Federated learning (FL) encounters substantial challenges due to heterogeneity, leading to gradient noise, client drift, and partial client participation errors, the last of which is the most pervasive but remains insufficiently addressed in current literature. In this paper, we propose FedAdaVR, a novel FL algorithm aimed at solving heterogeneity issues caused by sporadic client participation by incorporating an adaptive optimiser with a variance reduction technique. This method takes advantage of the most recent stored updates from clients, even when they are absent from the current training round, thereby emulating their presence. Furthermore, we propose FedAdaVR-Quant, which stores client updates in quantised form, significantly reducing the memory requirements (by 50%, 75%, and 87.5%) of FedAdaVR while maintaining equivalent model performance. We analyse the convergence behaviour of FedAdaVR under general nonconvex conditions and prove that our proposed algorithm can eliminate partial client participation error. Extensive experiments conducted on multiple datasets, under both independent and identically distributed (IID) and non-IID settings, demonstrate that FedAdaVR consistently outperforms state-of-the-art baseline methods.

翻译：联邦学习（FL）因异构性面临重大挑战，导致梯度噪声、客户端漂移和部分客户端参与误差，其中后者最为普遍，但在现有文献中仍未得到充分解决。本文提出FedAdaVR，一种新颖的FL算法，通过将自适应优化器与方差缩减技术相结合，旨在解决因客户端偶发性参与导致的异构性问题。该方法利用客户端最近存储的更新，即使这些客户端未参与当前训练轮次，从而模拟其参与。此外，我们提出FedAdaVR-Quant，以量化形式存储客户端更新，在保持同等模型性能的同时，显著降低FedAdaVR的内存需求（分别降低50%、75%和87.5%）。我们分析了FedAdaVR在一般非凸条件下的收敛行为，并证明所提算法能够消除部分客户端参与误差。在多个数据集上，于独立同分布（IID）和非IID设置下进行的广泛实验表明，FedAdaVR始终优于最先进的基线方法。