Federated Learning (FL) is a popular approach for distributed deep learning that prevents the pooling of large amounts of data in a central server. FL relies on clients to update a global model using their local datasets. Classical FL algorithms use a central federator that, for each training round, waits for all clients to send their model updates before aggregating them. In practical deployments, clients might have different computing powers and network capabilities, which might lead slow clients to become performance bottlenecks. Previous works have suggested to use a deadline for each learning round so that the federator ignores the late updates of slow clients, or so that clients send partially trained models before the deadline. To speed up the training process, we instead propose Aergia, a novel approach where slow clients (i) freeze the part of their model that is the most computationally intensive to train; (ii) train the unfrozen part of their model; and (iii) offload the training of the frozen part of their model to a faster client that trains it using its own dataset. The offloading decisions are orchestrated by the federator based on the training speed that clients report and on the similarities between their datasets, which are privately evaluated thanks to a trusted execution environment. We show through extensive experiments that Aergia maintains high accuracy and significantly reduces the training time under heterogeneous settings by up to 27% and 53% compared to FedAvg and TiFL, respectively.

翻译：联邦学习（FL）是一种流行的分布式深度学习方法，可避免在中央服务器中汇集大量数据。FL依赖客户端使用其本地数据集更新全局模型。经典FL算法使用中央聚合器，在每个训练轮次中等待所有客户端发送模型更新后再进行聚合。在实际部署中，客户端可能具有不同的计算能力和网络性能，这可能导致慢速客户端成为性能瓶颈。先前的研究建议为每个学习轮次设定截止时间，使聚合器忽略慢速客户端的延迟更新，或让客户端在截止时间前发送部分训练的模型。为加速训练过程，我们提出Aergia这一新颖方法：慢速客户端（i）冻结模型中计算最密集的部分；（ii）训练模型中未冻结的部分；以及（iii）将冻结部分的训练任务卸载至更快的客户端，由其使用自身数据集进行训练。卸载决策由聚合器根据客户端报告的训练速度及其数据集之间的相似性（通过可信执行环境进行隐私评估）协调。大量实验表明，在异构设置下，Aergia可保持高精度，并显著降低训练时间：相比FedAvg和TiFL，分别最高减少27%和53%。