The increasing demand for intelligent services and privacy protection of mobile and Internet of Things (IoT) devices motivates the wide application of Federated Edge Learning (FEL), in which devices collaboratively train on-device Machine Learning (ML) models without sharing their private data. Limited by device hardware, diverse user behaviors and network infrastructure, the algorithm design of FEL faces challenges related to resources, personalization and network environments. Fortunately, Knowledge Distillation (KD) has been leveraged as an important technique to tackle the above challenges in FEL. In this paper, we investigate the works that KD applies to FEL, discuss the limitations and open problems of existing KD-based FEL approaches, and provide guidance for their real deployment.

翻译：移动和物联网设备对智能服务及隐私保护日益增长的需求，推动了联邦边缘学习的广泛应用。在该框架中，设备通过协作训练本地机器学习模型，同时无需共享私有数据。受设备硬件、用户行为多样性及网络基础设施的限制，联邦边缘学习的算法设计面临资源、个性化及网络环境等方面的挑战。幸运的是，知识蒸馏已被用作解决联邦边缘学习中上述挑战的重要技术。本文研究了知识蒸馏在联邦边缘学习中的应用工作，探讨了现有基于知识蒸馏的联邦边缘学习方法的局限性及开放问题，并为其实际部署提供了指导。