Federated learning (FL) aims to train machine learning (ML) models across potentially millions of edge client devices. Yet, training and customizing models for FL clients is notoriously challenging due to the heterogeneity of client data, device capabilities, and the massive scale of clients, making individualized model exploration prohibitively expensive. State-of-the-art FL solutions personalize a globally trained model or concurrently train multiple models, but they often incur suboptimal model accuracy and huge training costs. In this paper, we introduce FedTrans, a multi-model FL training framework that automatically produces and trains high-accuracy, hardware-compatible models for individual clients at scale. FedTrans begins with a basic global model, identifies accuracy bottlenecks in model architectures during training, and then employs model transformation to derive new models for heterogeneous clients on the fly. It judiciously assigns models to individual clients while performing soft aggregation on multi-model updates to minimize total training costs. Our evaluations using realistic settings show that FedTrans improves individual client model accuracy by 14% - 72% while slashing training costs by 1.6X - 20X over state-of-the-art solutions.

翻译：联邦学习（FL）旨在跨潜在数百万边缘客户端设备训练机器学习（ML）模型。然而，由于客户端数据的异质性、设备能力的差异以及客户端规模的庞大性，为FL客户端训练和定制模型极具挑战性，使得个性化模型探索的成本过高。现有最先进的FL方案要么对全局训练模型进行个性化调整，要么同时训练多个模型，但这些方法常导致模型精度次优且训练成本高昂。本文提出FedTrans——一个多模型FL训练框架，能够自动为大规模个体客户端生成并训练高精度、硬件兼容的模型。FedTrans从基础全局模型出发，在训练过程中识别模型架构中的精度瓶颈，随后通过模型转换动态为异构客户端派生新模型。它通过智能地将模型分配给个体客户端，同时对多模型更新进行软聚合，以最小化总训练成本。基于真实场景的评估表明，与现有最先进方案相比，FedTrans将个体客户端模型精度提升了14%-72%，同时将训练成本削减了1.6倍至20倍。