While federated learning is promising for privacy-preserving collaborative learning without revealing local data, it remains vulnerable to white-box attacks and struggles to adapt to heterogeneous clients. Federated distillation (FD), built upon knowledge distillation--an effective technique for transferring knowledge from a teacher model to student models--emerges as an alternative paradigm, which provides enhanced privacy guarantees and addresses model heterogeneity. Nevertheless, challenges arise due to variations in local data distributions and the absence of a well-trained teacher model, which leads to misleading and ambiguous knowledge sharing that significantly degrades model performance. To address these issues, this paper proposes a selective knowledge sharing mechanism for FD, termed Selective-FD. It includes client-side selectors and a server-side selector to accurately and precisely identify knowledge from local and ensemble predictions, respectively. Empirical studies, backed by theoretical insights, demonstrate that our approach enhances the generalization capabilities of the FD framework and consistently outperforms baseline methods.

翻译：联邦学习虽能在不泄露本地数据的情况下实现隐私保护协同学习，但易受白盒攻击且难以适配异构客户端。基于知识蒸馏（一种将知识从教师模型迁移至学生模型的有效技术）的联邦蒸馏（FD）作为替代范式应运而生，其能提供更强的隐私保障并解决模型异构性问题。然而，由于本地数据分布差异及缺乏训练完备的教师模型，会导致误导性、模糊化的知识共享，显著降低模型性能。针对上述问题，本文提出面向联邦蒸馏的选择性知识共享机制Selective-FD，该机制包含客户端选择器与服务器端选择器，分别用于从本地预测与集成预测中精确识别有效知识。实证研究与理论分析表明，本方法可增强联邦蒸馏框架的泛化能力，且持续优于基线方法。