The increasing demand for intelligent services and privacy protection of mobile and Internet of Things (IoT) devices motivates the wide application of Federated Edge Learning (FEL), in which devices collaboratively train on-device Machine Learning (ML) models without sharing their private data. Limited by device hardware, diverse user behaviors and network infrastructure, the algorithm design of FEL faces challenges related to resources, personalization and network environments. Fortunately, Knowledge Distillation (KD) has been leveraged as an important technique to tackle the above challenges in FEL. In this paper, we investigate the works that KD applies to FEL, discuss the limitations and open problems of existing KD-based FEL approaches, and provide guidance for their real deployment.

翻译：随着移动和物联网设备对智能服务及隐私保护需求的日益增长，联邦边缘学习（Federated Edge Learning, FEL）得到广泛应用，其中设备协同训练本地机器学习模型而无需共享私有数据。受设备硬件、用户行为多样性和网络基础设施的限制，FEL算法设计面临资源、个性化和网络环境等方面的挑战。幸运的是，知识蒸馏（Knowledge Distillation, KD）已成为应对上述FEL挑战的重要技术。本文系统梳理了KD在FEL中的应用研究，探讨现有基于KD的FEL方法的局限性与未解决问题，并为其实际部署提供指导。