Concerned with user data privacy, this paper presents a new federated learning (FL) method that trains machine learning models on edge devices without accessing sensitive data. Traditional FL methods, although privacy-protective, fail to manage model heterogeneity and incur high communication costs due to their reliance on aggregation methods. To address this limitation, we propose a resource-aware FL method that aggregates local knowledge from edge models and distills it into robust global knowledge through knowledge distillation. This method allows efficient multi-model knowledge fusion and the deployment of resource-aware models while preserving model heterogeneity. Our method improves communication cost and performance in heterogeneous data and models compared to existing FL algorithms. Notably, it reduces the communication cost of ResNet-32 by up to 50\% and VGG-11 by up to 10$\times$ while delivering superior performance.

翻译：面向用户数据隐私保护，本文提出一种新型联邦学习方法，该方法可在不访问敏感数据的前提下，于边缘设备上训练机器学习模型。传统联邦学习方法虽具备隐私保护特性，但无法处理模型异构性，且因依赖聚合方法而面临高昂通信成本。为解决这一局限，我们提出一种资源感知型联邦学习方法，通过从边缘模型中提取局部知识并经由知识蒸馏将其凝练为稳健的全局知识。该方法在保持模型异构性的同时，实现了高效的多模型知识融合与资源感知型模型的部署。与现有联邦学习算法相比，我们的方法在异构数据与模型场景下显著降低了通信成本并提升了性能。值得注意的是，该方法在保持优越性能的同时，使ResNet-32的通信成本降低50%以上，使VGG-11的通信成本降低10倍。