Federated distillation has emerged as a promising collaborative machine learning approach, offering enhanced privacy protection and reduced communication compared to traditional federated learning by exchanging model outputs (soft logits) rather than full model parameters. However, existing methods employ complex selective knowledge-sharing strategies that require clients to identify in-distribution proxy data through computationally expensive statistical density ratio estimators. Additionally, server-side filtering of ambiguous knowledge introduces latency to the process. To address these challenges, we propose a robust, resource-efficient EdgeFD method that reduces the complexity of the client-side density ratio estimation and removes the need for server-side filtering. EdgeFD introduces an efficient KMeans-based density ratio estimator for effectively filtering both in-distribution and out-of-distribution proxy data on clients, significantly improving the quality of knowledge sharing. We evaluate EdgeFD across diverse practical scenarios, including strong non-IID, weak non-IID, and IID data distributions on clients, without requiring a pre-trained teacher model on the server for knowledge distillation. Experimental results demonstrate that EdgeFD outperforms state-of-the-art methods, consistently achieving accuracy levels close to IID scenarios even under heterogeneous and challenging conditions. The significantly reduced computational overhead of the KMeans-based estimator is suitable for deployment on resource-constrained edge devices, thereby enhancing the scalability and real-world applicability of federated distillation. The code is available online for reproducibility.

翻译：联邦蒸馏作为一种协作式机器学习方法，通过交换模型输出（软逻辑值）而非完整模型参数，在增强隐私保护、降低通信开销方面展现出传统联邦学习所不具备的优势。然而现有方法采用复杂的知识选择性共享策略，需要客户端通过计算开销昂贵的统计密度比估计器来识别分布内代理数据。此外，服务端对模糊知识的过滤会引入处理延迟。针对上述问题，我们提出一种鲁棒高效的边缘联邦蒸馏方法（EdgeFD），该方法在降低客户端密度比估计复杂度的同时，消除了服务端过滤需求。EdgeFD引入基于KMeans的高效密度比估计器，在客户端有效过滤分布内与分布外代理数据，显著提升知识共享质量。我们在包含强非独立同分布、弱非独立同分布及独立同分布数据的多样化实际场景中评估EdgeFD，且无需在服务端预训练教师模型进行知识蒸馏。实验结果表明，即使在异质性和挑战性条件下，EdgeFD仍持续超越现有最优方法，其精度水平接近独立同分布场景。基于KMeans估计器的计算开销显著降低，使其适用于资源受限的边缘设备部署，从而增强联邦蒸馏的可扩展性与现实应用性。相关代码已开源供可重复研究使用。