In cross-device Federated Learning (FL) environments, scaling synchronous FL methods is challenging as stragglers hinder the training process. Moreover, the availability of each client to join the training is highly variable over time due to system heterogeneities and intermittent connectivity. Recent asynchronous FL methods (e.g., FedBuff) have been proposed to overcome these issues by allowing slower users to continue their work on local training based on stale models and to contribute to aggregation when ready. However, we show empirically that this method can lead to a substantial drop in training accuracy as well as a slower convergence rate. The primary reason is that fast-speed devices contribute to many more rounds of aggregation while others join more intermittently or not at all, and with stale model updates. To overcome this barrier, we propose TimelyFL, a heterogeneity-aware asynchronous FL framework with adaptive partial training. During the training, TimelyFL adjusts the local training workload based on the real-time resource capabilities of each client, aiming to allow more available clients to join in the global update without staleness. We demonstrate the performance benefits of TimelyFL by conducting extensive experiments on various datasets (e.g., CIFAR-10, Google Speech, and Reddit) and models (e.g., ResNet20, VGG11, and ALBERT). In comparison with the state-of-the-art (i.e., FedBuff), our evaluations reveal that TimelyFL improves participation rate by 21.13%, harvests 1.28x - 2.89x more efficiency on convergence rate, and provides a 6.25% increment on test accuracy.

翻译：在跨设备联邦学习环境中，扩展同步联邦学习方法面临挑战，因为掉队者会阻碍训练进程。此外，由于系统异构性和间歇性连接，每个客户端参与训练的可用性随时间高度变化。近期提出的异步联邦学习方法（如FedBuff）通过允许较慢用户基于陈旧模型继续本地训练，并在准备就绪时参与聚合来克服这些问题。然而，我们的实证研究表明，该方法可能导致训练准确率大幅下降以及收敛速度减慢。主要原因是快速设备贡献了更多轮次的聚合，而其他设备则更间歇性地参与或根本不参与，且携带陈旧模型更新。为突破这一障碍，我们提出TimelyFL——一种基于自适应部分训练的异构性感知异步联邦学习框架。在训练过程中，TimelyFL根据每个客户端的实时资源能力调整本地训练工作量，旨在让更多可用客户端在不产生模型陈旧性的情况下参与全局更新。我们在多种数据集（如CIFAR-10、Google Speech和Reddit）和模型（如ResNet20、VGG11和ALBERT）上进行了广泛实验，验证了TimelyFL的性能优势。与现有最优方法（即FedBuff）相比，我们的评估表明，TimelyFL将参与率提升了21.13%，收敛效率提高了1.28倍至2.89倍，测试准确率提升了6.25%。