Federated learning is a new distributed machine learning framework, where a bunch of heterogeneous clients collaboratively train a model without sharing training data. In this work, we consider a practical and ubiquitous issue in federated learning: intermittent client availability, where the set of eligible clients may change during the training process. Such an intermittent client availability model would significantly deteriorate the performance of the classical Federated Averaging algorithm (FedAvg for short). We propose a simple distributed non-convex optimization algorithm, called Federated Latest Averaging (FedLaAvg for short), which leverages the latest gradients of all clients, even when the clients are not available, to jointly update the global model in each iteration. Our theoretical analysis shows that FedLaAvg attains the convergence rate of $O(1/(N^{1/4} T^{1/2}))$, achieving a sublinear speedup with respect to the total number of clients. We implement and evaluate FedLaAvg with the CIFAR-10 dataset. The evaluation results demonstrate that FedLaAvg indeed reaches a sublinear speedup and achieves 4.23% higher test accuracy than FedAvg.

翻译：联邦学习是一个新的分布式机器学习框架,在这个框架中,一大批各式各样的客户在不分享培训数据的情况下合作培训一个模型。在这项工作中,我们考虑到联合学习中一个实际和普遍的问题:间歇性客户的可用性,合格客户群在培训过程中可能会发生变化。这种间歇性客户可用性模式将大大降低古典联邦异变算法(FedAvg为短)的性能。我们建议一个简单的分布式非康氏优化算法,称为FedLaAvg为短,它利用所有客户的最新梯度,即使客户没有这种梯度,在每次循环中共同更新全球模型。我们的理论分析表明,FedLaavg达到美元(1/N ⁇ 1/4}T ⁇ 1/2})的趋同率,在客户总数方面实现亚线性速度的子加速。我们用CIFAR-10数据集来实施和评价FedLaAvg。评价结果表明,FedLavg的确达到了亚弗的亚氏子线性速度并达到4.23%的精度测试。