Federated learning (FL) is a promising framework for privacy-preserving collaborative learning. In FL, the model training tasks are distributed to clients and only the model updates need to be collected at a central server. However, when being deployed at the mobile edge network, clients (e.g., smartphones and wearables) may have unpredictable availability and randomly drop out of any training iteration, which hinders FL from achieving the convergence. This paper tackles such a critical challenge of FL. In particular, we first investigate the convergence of the classical FedAvg algorithm with arbitrary client dropouts. We find that with the common choice of a decaying learning rate, FedAvg can only oscillate within the neighborhood of a stationary point of the global loss function, which is caused by the divergence between the aggregated update and the desired central update. Motivated by this new observation, we then design a novel training algorithm named MimiC, where the server modifies each received model update based on the previous ones. The proposed modification of the received model updates is able to mimic the imaginary central update irrespective of the dropout clients. The theoretical analysis of MimiC shows that the divergence between the aggregated update and the central update diminishes with a proper choice of the learning rates, leading to its convergence. Simulation results further demonstrate that MimiC maintains stable convergence performance in the presence of client dropouts and learns better models than the baseline methods.

翻译：联邦学习（FL）是一种面向隐私保护协同学习的有前景框架。在FL中，模型训练任务被分配给客户端，仅在中央服务器收集模型更新。然而，当部署于移动边缘网络时，客户端（如智能手机和可穿戴设备）可能面临不可预见的可用性问题，并随机退出任意训练迭代，这阻碍了FL实现收敛。本文致力于解决这一关键挑战。具体而言，我们首先研究了经典FedAvg算法在存在任意客户端丢失情况下的收敛性。研究发现，在采用常见的衰减学习率时，FedAvg仅能在全局损失函数驻点邻域内振荡，其根源在于聚合更新与期望的中央更新之间存在偏差。受这一新发现的启发，我们随后设计了一种名为MimiC的新型训练算法，其中服务器基于历史接收的模型更新对当前接收的更新进行修正。所提出的修正方法能够无视丢失客户端，有效模拟虚拟的中央更新。MimiC的理论分析表明，通过合理选择学习率，聚合更新与中心更新之间的偏差会逐渐消失，从而保证其收敛性。仿真结果进一步证明，MimiC在客户端丢失情况下仍能保持稳定的收敛性能，并学习到优于基线方法的模型。