Personalized federated learning (pFL) enables collaborative training among multiple clients to enhance the capability of customized local models. In pFL, clients may have heterogeneous (also known as non-IID) data, which poses a key challenge in how to decouple the data knowledge into generic knowledge for global sharing and personalized knowledge for preserving local personalization. A typical way of pFL focuses on label distribution skew, and they adopt a decoupling scheme where the model is split into a common feature extractor and two prediction heads (generic and personalized). However, such a decoupling scheme cannot solve the essential problem of feature skew heterogeneity, because a common feature extractor cannot decouple the generic and personalized features. Therefore, in this paper, we rethink the architecture decoupling design for feature-skew pFL and propose an effective pFL method called FediOS. In FediOS, we reformulate the decoupling into two feature extractors (generic and personalized) and one shared prediction head. Orthogonal projections are used for clients to map the generic features into one common subspace and scatter the personalized features into different subspaces to achieve decoupling for them. In addition, a shared prediction head is trained to balance the importance of generic and personalized features during inference. Extensive experiments on four vision datasets demonstrate our method reaches state-of-the-art pFL performances under feature skew heterogeneity.

翻译：个性化联邦学习(pFL)使多个客户端能够协同训练以增强定制化本地模型的能力。在pFL中，客户端可能拥有异构（即非独立同分布）数据，这带来的关键挑战在于如何将数据知识解耦为用于全局共享的通用知识和用于保持本地个性化的个性化知识。典型pFL方法聚焦于标签分布偏斜问题，采用将模型分为通用特征提取器与两个预测头（通用和个性化）的解耦方案。然而，这种解耦方案无法解决特征偏斜异质性的本质问题，因为单一通用特征提取器无法同时解耦通用特征与个性化特征。为此，本文重新思考面向特征偏斜pFL的架构解耦设计，提出名为FediOS的高效pFL方法。在FediOS中，我们将解耦重构为两个特征提取器（通用与个性化）与一个共享预测头。通过正交投影，客户端将通用特征映射到共同子空间，并将个性化特征分散到不同子空间，从而实现两类特征的解耦。此外，训练共享预测头以在推理过程中平衡通用特征与个性化特征的重要性。在四个视觉数据集上的大量实验表明，本方法在特征偏斜异质性场景下达到了最先进的pFL性能。