Federated learning are inherently hampered by data heterogeneity: non-iid distributed training data over local clients. We propose a novel model training approach for federated learning, FLex&Chill, which exploits the Logit Chilling method. Through extensive evaluations, we demonstrate that, in the presence of non-iid data characteristics inherent in federated learning systems, this approach can expedite model convergence and improve inference accuracy. Quantitatively, from our experiments, we observe up to 6X improvement in the global federated learning model convergence time, and up to 3.37% improvement in inference accuracy.
翻译:联邦学习本质上面临数据异构性的阻碍:本地客户端上的非独立同分布训练数据。我们提出了一种新颖的联邦学习模型训练方法FLex&Chill,该方法利用了Logit冷却技术。通过广泛的评估,我们证明在联邦学习系统固有的非独立同分布数据特性下,该方法能够加速模型收敛并提升推理精度。定量实验结果表明,全局联邦学习模型的收敛时间最多可提升6倍,推理精度最多可提升3.37%。