Edge Intelligence (EI) allows Artificial Intelligence (AI) applications to run at the edge, where data analysis and decision-making can be performed in real-time and close to data sources. To protect data privacy and unify data silos among end devices in EI, Federated Learning (FL) is proposed for collaborative training of shared AI models across devices without compromising data privacy. However, the prevailing FL approaches cannot guarantee model generalization and adaptation on heterogeneous clients. Recently, Personalized Federated Learning (PFL) has drawn growing awareness in EI, as it enables a productive balance between local-specific training requirements inherent in devices and global-generalized optimization objectives for satisfactory performance. However, most existing PFL methods are based on the Parameters Interaction-based Architecture (PIA) represented by FedAvg, which causes unaffordable communication burdens due to large-scale parameters transmission between devices and the edge server. In contrast, Logits Interaction-based Architecture (LIA) allows to update model parameters with logits transfer and gains the advantages of communication lightweight and heterogeneous on-device model allowance compared to PIA. Nevertheless, previous LIA methods attempt to achieve satisfactory performance either relying on unrealistic public datasets or increasing communication overhead for additional information transmission other than logits. To tackle this dilemma, we propose a knowledge cache-driven PFL architecture, named FedCache, which reserves a knowledge cache on the server for fetching personalized knowledge from the samples with similar hashes to each given on-device sample. During the training phase, ensemble distillation is applied to on-device models for constructive optimization with personalized knowledge transferred from the server-side knowledge cache.

翻译：边缘智能（EI）使人工智能（AI）应用能够在边缘运行，从而在靠近数据源的位置实时进行数据分析与决策。为保护EI中终端设备的数据隐私并打破数据孤岛，联邦学习（FL）被提出用于跨设备协作训练共享AI模型，且不损害数据隐私。然而，现有主流FL方法无法保证模型在异构客户端上的泛化性与适应性。近年来，个性化联邦学习（PFL）在EI领域日益受到关注，因为它能在设备固有的本地特定训练需求与全局泛化优化目标之间实现有效平衡，从而获得令人满意的性能。但现有多数PFL方法基于以FedAvg为代表的参数交互架构（PIA），由于设备与边缘服务器之间需传输大规模参数，导致其通信开销难以承受。相比之下，逻辑交互架构（LIA）通过逻辑传递更新模型参数，相比PIA具有通信轻量化和允许设备端模型异构的优势。然而，现有LIA方法要么依赖不切实际的公共数据集，要么通过增加额外信息传输（除逻辑外）提升通信开销来获得满意性能。为应对这一困境，我们提出了一种名为FedCache的知识缓存驱动PFL架构。该架构在服务器端维护知识缓存，通过为每个设备端样本提取哈希值相近的样本中的个性化知识。在训练阶段，利用集成蒸馏方法，结合从服务器端知识缓存传递的个性化知识，对设备端模型进行建设性优化。