Federated learning (FL) is a privacy-preserving collaboratively machine learning paradigm. Traditional FL requires all data owners (a.k.a. FL clients) to train the same local model. This design is not well-suited for scenarios involving data and/or system heterogeneity. Model-Heterogeneous Personalized FL (MHPFL) has emerged to address this challenge. Existing MHPFL approaches often rely on a public dataset with the same nature as the learning task, or incur high computation and communication costs. To address these limitations, we propose the Federated Semantic Similarity Aggregation (FedSSA) approach for supervised classification tasks, which splits each client's model into a heterogeneous (structure-different) feature extractor and a homogeneous (structure-same) classification header. It performs local-to-global knowledge transfer via semantic similarity-based header parameter aggregation. In addition, global-to-local knowledge transfer is achieved via an adaptive parameter stabilization strategy which fuses the seen-class parameters of historical local headers with that of the latest global header for each client. FedSSA does not rely on public datasets, while only requiring partial header parameter transmission to save costs. Theoretical analysis proves the convergence of FedSSA. Extensive experiments present that FedSSA achieves up to 3.62% higher accuracy, 15.54 times higher communication efficiency, and 15.52 times higher computational efficiency compared to 7 state-of-the-art MHPFL baselines.

翻译：联邦学习（FL）是一种隐私保护的协同机器学习范式。传统FL要求所有数据所有者（即FL客户端）训练相同的本地模型，这种设计难以适用于涉及数据和/或系统异构性的场景。模型异质个性化联邦学习（MHPFL）应运而生以解决该挑战。现有MHPFL方法往往依赖与学习任务同质的公开数据集，或产生高昂的计算和通信成本。为克服这些局限，我们针对有监督分类任务提出联邦语义相似度聚合（FedSSA）方法，该方法将每个客户端的模型拆分为异构（结构不同）特征提取器与同构（结构相同）分类头，通过基于语义相似度的分类头参数聚合实现局部到全局的知识迁移。此外，通过自适应参数稳定化策略实现全局到局部的知识迁移，该策略为每个客户端融合历史局部分类头的已见类参数与最新全局分类头的对应参数。FedSSA不依赖公开数据集，且仅需传输部分分类头参数以节省成本。理论分析证明了FedSSA的收敛性。大量实验表明，与7个最先进的MHPFL基线相比，FedSSA在精度上最高提升3.62%，通信效率提升15.54倍，计算效率提升15.52倍。