Federated Learning is a training framework that enables multiple participants to collaboratively train a shared model while preserving data privacy and minimizing communication overhead. The heterogeneity of devices and networking resources of the participants delay the training and aggregation in federated learning. This paper proposes a federated learning approach to manoeuvre the heterogeneity among the participants using resource aware clustering. The approach begins with the server gathering information about the devices and networking resources of participants, after which resource aware clustering is performed to determine the optimal number of clusters using Dunn Indices. The mechanism of participant assignment is then introduced, and the expression of communication rounds required for model convergence in each cluster is mathematically derived. Furthermore, a master-slave technique is introduced to improve the performance of the lightweight models in the clusters using knowledge distillation. Finally, experimental evaluations are conducted to verify the feasibility and effectiveness of the approach and to compare it with state-of-the-art techniques.
翻译:联邦学习是一种训练框架,允许多个参与者协同训练共享模型,同时保护数据隐私并最小化通信开销。参与者的设备与网络资源异质性会延迟联邦学习中的训练与聚合过程。本文提出一种通过资源感知聚类来应对参与者异质性的联邦学习方法。该方法首先由服务器收集参与者的设备与网络资源信息,随后基于邓恩指数执行资源感知聚类以确定最优聚类数量。接着引入参与者分配机制,并从数学上推导出每个聚类中模型收敛所需的通信轮次表达式。此外,引入主从技术,通过知识蒸馏提升聚类中轻量级模型的性能。最后,通过实验评估验证该方法的可行性与有效性,并与现有先进技术进行对比。