We present a partially personalized formulation of Federated Learning (FL) that strikes a balance between the flexibility of personalization and cooperativeness of global training. In our framework, we split the variables into global parameters, which are shared across all clients, and individual local parameters, which are kept private. We prove that under the right split of parameters, it is possible to find global parameters that allow each client to fit their data perfectly, and refer to the obtained problem as overpersonalized. For instance, the shared global parameters can be used to learn good data representations, whereas the personalized layers are fine-tuned for a specific client. Moreover, we present a simple algorithm for the partially personalized formulation that offers significant benefits to all clients. In particular, it breaks the curse of data heterogeneity in several settings, such as training with local steps, asynchronous training, and Byzantine-robust training.

翻译：我们提出了一种部分个性化的联邦学习（FL）框架，在个性化灵活性与全局训练的协作性之间取得了平衡。在该框架中，我们将变量分为客户端间共享的全局参数和保持私有的个体本地参数。我们证明，在正确的参数划分下，能够找到使每个客户端完美拟合其数据的全局参数，并将所得问题称为过个性化问题。例如，共享全局参数可用于学习良好的数据表示，而个性化层则针对特定客户端进行微调。此外，我们为该部分个性化公式提出了一种简单算法，能为所有客户端带来显著收益。具体而言，该算法在多种场景下突破了数据异构性困境，包括本地步骤训练、异步训练以及拜占庭鲁棒训练。