Federated learning enables distributed clients to collaborate on training while storing their data locally to protect client privacy. However, due to the heterogeneity of data, models, and devices, the final global model may need to perform better for tasks on each client. Communication bottlenecks, data heterogeneity, and model heterogeneity have been common challenges in federated learning. In this work, we considered a label distribution skew problem, a type of data heterogeneity easily overlooked. In the context of classification, we propose a personalized federated learning approach called pFedPM. In our process, we replace traditional gradient uploading with feature uploading, which helps reduce communication costs and allows for heterogeneous client models. These feature representations play a role in preserving privacy to some extent. We use a hyperparameter $a$ to mix local and global features, which enables us to control the degree of personalization. We also introduced a relation network as an additional decision layer, which provides a non-linear learnable classifier to predict labels. Experimental results show that, with an appropriate setting of $a$, our scheme outperforms several recent FL methods on MNIST, FEMNIST, and CRIFAR10 datasets and achieves fewer communications.

翻译：联邦学习使得分布式客户端能够在本地存储数据的同时协作训练，从而保护客户端隐私。然而，由于数据、模型和设备的异质性，最终的全局模型可能无法在每个客户端的任务上取得良好性能。通信瓶颈、数据异质性和模型异质性一直是联邦学习中常见的挑战。在本研究中，我们考虑了一种容易被忽视的数据异质性问题——标签分布偏移问题。在分类任务背景下，我们提出了一种名为pFedPM的个性化联邦学习方法。在我们的流程中，我们使用特征上传替代传统的梯度上传，这有助于降低通信成本并允许客户端模型具有异质性。这些特征表示在一定程度上起到了保护隐私的作用。我们使用超参数$a$来混合本地特征和全局特征，从而能够控制个性化程度。我们还引入了一个关系网络作为额外的决策层，该网络提供了一个非线性可学习的分类器来预测标签。实验结果表明，在适当设置$a$的情况下，我们的方案在MNIST、FEMNIST和CRIFAR10数据集上优于多种近期联邦学习方法，并且实现了更少的通信次数。