In recent years, federated learning (FL) has emerged as a promising technique for training machine learning models in a decentralized manner while also preserving data privacy. The non-independent and identically distributed (non-i.i.d.) nature of client data, coupled with constraints on client or edge devices, presents significant challenges in FL. Furthermore, learning across a high number of communication rounds can be risky and potentially unsafe for model exploitation. Traditional FL approaches may suffer from these challenges. Therefore, we introduce FedSiKD, which incorporates knowledge distillation (KD) within a similarity-based federated learning framework. As clients join the system, they securely share relevant statistics about their data distribution, promoting intra-cluster homogeneity. This enhances optimization efficiency and accelerates the learning process, effectively transferring knowledge between teacher and student models and addressing device constraints. FedSiKD outperforms state-of-the-art algorithms by achieving higher accuracy, exceeding by 25\% and 18\% for highly skewed data at $\alpha = {0.1,0.5}$ on the HAR and MNIST datasets, respectively. Its faster convergence is illustrated by a 17\% and 20\% increase in accuracy within the first five rounds on the HAR and MNIST datasets, respectively, highlighting its early-stage learning proficiency. Code is publicly available and hosted on GitHub (https://github.com/SimuEnv/FedSiKD)

翻译：近年来，联邦学习作为一种在保护数据隐私的同时以去中心化方式训练机器学习模型的技术崭露头角。客户端数据的非独立同分布特性，加之客户端或边缘设备的计算约束，给联邦学习带来了重大挑战。此外，在大量通信轮次中进行学习可能伴随风险，且潜在威胁模型安全。传统联邦学习方法难以应对这些挑战。为此，我们提出FedSiKD，它在基于相似度的联邦学习框架中融入了知识蒸馏技术。当客户端加入系统时，它们安全地共享关于其数据分布的相关统计信息，从而促进簇内同质性。这提升了优化效率并加速了学习过程，有效实现了师生模型间的知识迁移，并解决了设备约束问题。FedSiKD在高度倾斜数据上取得了更优性能：在α=0.1和0.5的参数下，对于HAR和MNIST数据集，其准确率分别超出最先进算法25%和18%。其快速收敛特性体现在：在HAR和MNIST数据集的前五轮训练中，准确率分别提升17%和20%，凸显了早期学习的高效性。代码已开源并托管于GitHub（https://github.com/SimuEnv/FedSiKD）。