The growing interest in intelligent services and privacy protection for mobile devices has given rise to the widespread application of federated learning in Multi-access Edge Computing (MEC). Diverse user behaviors call for personalized services with heterogeneous Machine Learning (ML) models on different devices. Federated Multi-task Learning (FMTL) is proposed to train related but personalized ML models for different devices, whereas previous works suffer from excessive communication overhead during training and neglect the model heterogeneity among devices in MEC. Introducing knowledge distillation into FMTL can simultaneously enable efficient communication and model heterogeneity among clients, whereas existing methods rely on a public dataset, which is impractical in reality. To tackle this dilemma, Federated MultI-task Distillation for Multi-access Edge CompuTing (FedICT) is proposed. FedICT direct local-global knowledge aloof during bi-directional distillation processes between clients and the server, aiming to enable multi-task clients while alleviating client drift derived from divergent optimization directions of client-side local models. Specifically, FedICT includes Federated Prior Knowledge Distillation (FPKD) and Local Knowledge Adjustment (LKA). FPKD is proposed to reinforce the clients' fitting of local data by introducing prior knowledge of local data distributions. Moreover, LKA is proposed to correct the distillation loss of the server, making the transferred local knowledge better match the generalized representation. Experiments on three datasets show that FedICT significantly outperforms all compared benchmarks in various data heterogeneous and model architecture settings, achieving improved accuracy with less than 1.2% training communication overhead compared with FedAvg and no more than 75% training communication round compared with FedGKT.

翻译：摘要：随着移动设备对智能服务与隐私保护需求的日益增长，联邦学习在多接入边缘计算（MEC）中得到广泛应用。多样化的用户行为要求在不同设备上部署异构机器学习（ML）模型以实现个性化服务。联邦多任务学习（FMTL）旨在为不同设备训练相关但个性化的ML模型，然而现有方法存在训练过程中通信开销过大，且未考虑MEC环境下设备间模型异构性的问题。将知识蒸馏引入FMTL可同时实现高效通信与客户端模型异构，但现有方法依赖公共数据集，这在现实中难以实现。为应对这一困境，本文提出面向多接入边缘计算的联邦多任务蒸馏方法（FedICT）。FedICT通过在客户端与服务器之间的双向蒸馏过程中实现本地-全局知识对齐，旨在赋能多任务客户端的同时，缓解因客户端本地模型优化方向差异导致的客户端漂移问题。具体而言，FedICT包含联邦先验知识蒸馏（FPKD）与本地知识调整（LKA）两个模块。FPKD通过引入本地数据分布的先验知识，增强客户端对本地数据的拟合能力。LKA则用于修正服务器的蒸馏损失，使迁移的本地知识更好地匹配全局表征。在三个数据集上的实验表明，FedICT在多种数据异构和模型架构设置下显著优于所有对比基准：相比FedAvg，其训练通信开销降低至1.2%以内即可提升精度；相比FedGKT，其训练通信轮次不超过75%。