This paper addresses the challenge of mitigating data heterogeneity among clients within a Federated Learning (FL) framework. The model-drift issue, arising from the noniid nature of client data, often results in suboptimal personalization of a global model compared to locally trained models for each client. To tackle this challenge, we propose a novel approach named FedD2S for Personalized Federated Learning (pFL), leveraging knowledge distillation. FedD2S incorporates a deep-to-shallow layer-dropping mechanism in the data-free knowledge distillation process to enhance local model personalization. Through extensive simulations on diverse image datasets-FEMNIST, CIFAR10, CINIC0, and CIFAR100-we compare FedD2S with state-of-the-art FL baselines. The proposed approach demonstrates superior performance, characterized by accelerated convergence and improved fairness among clients. The introduced layer-dropping technique effectively captures personalized knowledge, resulting in enhanced performance compared to alternative FL models. Moreover, we investigate the impact of key hyperparameters, such as the participation ratio and layer-dropping rate, providing valuable insights into the optimal configuration for FedD2S. The findings demonstrate the efficacy of adaptive layer-dropping in the knowledge distillation process to achieve enhanced personalization and performance across diverse datasets and tasks.

翻译：本文解决了在联邦学习（FL）框架中缓解客户端数据异构性的挑战。由客户端数据非独立同分布特性引发的模型漂移问题，通常会导致全局模型相较于各客户端本地训练模型的个性化表现次优。为应对这一挑战，我们提出了一种名为FedD2S的新型个性化联邦学习（pFL）方法，该方法利用知识蒸馏技术。FedD2S在无数据知识蒸馏过程中引入了一种从深层到浅层的层级丢弃机制，以增强本地模型的个性化能力。通过在多种图像数据集（FEMNIST、CIFAR10、CINIC0和CIFAR100）上的广泛仿真实验，我们将FedD2S与最先进的FL基线方法进行了比较。所提出的方法表现出卓越性能，具有加速收敛和提升客户端间公平性的特点。引入的层级丢弃技术有效捕捉了个性化知识，从而相较于其他FL模型取得了更优性能。此外，我们研究了参与比例和层级丢弃率等关键超参数的影响，为FedD2S的最优配置提供了有价值的见解。研究结果表明，在知识蒸馏过程中采用自适应层级丢弃，能够在多样化数据集和任务中实现更强的个性化能力与更优的性能表现。