Federated learning is a collaborative model training method that iterates model updates by multiple clients and aggregation of the updates by a central server. Device and statistical heterogeneity of participating clients cause significant performance degradation so that an appropriate aggregation weight should be assigned to each client in the aggregation phase of the server. To adjust the aggregation weights, this paper employs deep unfolding, which is known as the parameter tuning method that leverages both learning capability using training data like deep learning and domain knowledge. This enables us to directly incorporate the heterogeneity of the environment of interest into the tuning of the aggregation weights. The proposed approach can be combined with various federated learning algorithms. The results of numerical experiments indicate that a higher test accuracy for unknown class-balanced data can be obtained with the proposed method than that with conventional heuristic weighting methods. The proposed method can handle large-scale learning models with the aid of pretrained models such that it can perform practical real-world tasks. Convergence rate of federated learning algorithms with the proposed method is also provided in this paper.
翻译:联邦学习是一种协作式模型训练方法,通过多个客户端进行模型更新迭代,并由中央服务器聚合这些更新。参与客户端的设备异构性和统计异构性会导致显著的性能下降,因此服务器在聚合阶段应为每个客户端分配适当的聚合权重。为调整聚合权重,本文采用深度展开方法——这是一种结合深度学习对训练数据的学习能力与领域知识的参数调优技术。该方法能直接将目标环境的异构性纳入聚合权重的调优过程中。所提方法可与多种联邦学习算法相结合。数值实验结果表明,相较于传统启发式加权方法,所提方法在对未知类别平衡数据上能获得更高的测试准确率。借助预训练模型,该方法可处理大规模学习模型,从而执行实际应用场景中的任务。本文还提供了采用所提方法的联邦学习算法的收敛速率分析。