Personalized federated learning has received an upsurge of attention due to the mediocre performance of conventional federated learning (FL) over heterogeneous data. Unlike conventional FL which trains a single global consensus model, personalized FL allows different models for different clients. However, existing personalized FL algorithms only implicitly transfer the collaborative knowledge across the federation by embedding the knowledge into the aggregated model or regularization. We observed that this implicit knowledge transfer fails to maximize the potential of each client's empirical risk toward other clients. Based on our observation, in this work, we propose Personalized Global Federated Learning (PGFed), a novel personalized FL framework that enables each client to personalize its own global objective by explicitly and adaptively aggregating the empirical risks of itself and other clients. To avoid massive (O(N^2)) communication overhead and potential privacy leakage while achieving this, each client's risk is estimated through a first-order approximation for other clients' adaptive risk aggregation. On top of PGFed, we develop a momentum upgrade, dubbed PGFedMo, to more efficiently utilize clients' empirical risks. Our extensive experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods. The code is publicly available at https://github.com/ljaiverson/pgfed.

翻译：个性化联邦学习因传统联邦学习在异构数据上表现平平而备受关注。不同于传统联邦学习训练单一全局共识模型，个性化联邦学习允许为不同客户端设定不同模型。然而，现有个性化FL算法仅通过将知识嵌入聚合模型或正则化项，隐式地在联邦中传递协作知识。我们观察到，这种隐式知识传递未能最大化每个客户端经验风险对其他客户端的潜力。基于此观察，本文提出个性化全局联邦学习（PGFed），一种新型个性化FL框架，通过显式且自适应地聚合自身及其他客户端的经验风险，使每个客户端能够个性化其全局目标。为避免巨大的（O(N²)）通信开销和潜在的隐私泄露，每个客户端的风险通过一阶近似来估计其他客户端的自适应风险聚合。在PGFed基础上，我们开发了动量升级版PGFedMo，以更高效地利用客户端的经验风险。在四种数据集上的大量实验表明，PGFed在不同联邦设置下均较以往最先进方法有持续改进。代码已公开于https://github.com/ljaiverson/pgfed。