Federated Learning (FL) enables multiple devices to collaboratively train a shared model while ensuring data privacy. The selection of participating devices in each training round critically affects both the model performance and training efficiency, especially given the vast heterogeneity in training capabilities and data distribution across devices. To address these challenges, we introduce a novel device selection solution called FedRank, which is an end-to-end, ranking-based approach that is pre-trained by imitation learning against state-of-the-art analytical approaches. It not only considers data and system heterogeneity at runtime but also adaptively and efficiently chooses the most suitable clients for model training. Specifically, FedRank views client selection in FL as a ranking problem and employs a pairwise training strategy for the smart selection process. Additionally, an imitation learning-based approach is designed to counteract the cold-start issues often seen in state-of-the-art learning-based approaches. Experimental results reveal that \model~ boosts model accuracy by 5.2\% to 56.9\%, accelerates the training convergence up to $2.01 \times$ and saves the energy consumption up to $40.1\%$.
翻译:联邦学习(FL)使多个设备能够协作训练共享模型,同时确保数据隐私。每轮训练中参与设备的选择对模型性能和训练效率具有关键影响,尤其是在设备训练能力和数据分布存在显著异质性的情况下。针对这些挑战,我们提出了一种名为FedRank的新型设备选择方案,这是一种端到端、基于排序的方法,通过模仿学习预训练,可媲美最先进的分析方法。该方法不仅能在运行时考虑数据与系统异质性,还能自适应且高效地选取最适合模型训练的客户端。具体而言,FedRank将FL中的客户端选择问题视为排序任务,并采用成对训练策略实现智能选择过程。此外,我们设计了基于模仿学习的方法来缓解当前主流学习型方法中常见的冷启动问题。实验结果表明,\model~ 将模型准确率提升5.2%至56.9%,训练收敛速度提升至$2.01 \times$,并将能耗降低高达40.1%。