Personalized federated learning (PFL) aims to produce the greatest personalized model for each client to face an insurmountable problem--data heterogeneity in real FL systems. However, almost all existing works have to face large communication burdens and the risk of disruption if the central server fails. Only limited efforts have been used in a decentralized way but still suffers from inferior representation ability due to sharing the full model with its neighbors. Therefore, in this paper, we propose a personalized FL framework with a decentralized partial model training called DFedAlt. It personalizes the "right" components in the modern deep models by alternately updating the shared and personal parameters to train partially personalized models in a peer-to-peer manner. To further promote the shared parameters aggregation process, we propose DFedSalt integrating the local Sharpness Aware Minimization (SAM) optimizer to update the shared parameters. It adds proper perturbation in the direction of the gradient to overcome the shared model inconsistency across clients. Theoretically, we provide convergence analysis of both algorithms in the general non-convex setting for decentralized partial model training in PFL. Our experiments on several real-world data with various data partition settings demonstrate that (i) decentralized training is more suitable for partial personalization, which results in state-of-the-art (SOTA) accuracy compared with the SOTA PFL baselines; (ii) the shared parameters with proper perturbation make partial personalized FL more suitable for decentralized training, where DFedSalt achieves most competitive performance.

翻译：个性化联邦学习（PFL）旨在为每个客户端生成最优的个性化模型，以应对真实联邦学习系统中数据异构性这一难以克服的问题。然而，现有研究几乎都面临巨大的通信负担，且中央服务器一旦失效便存在中断风险。仅有少量工作采用去中心化方式，但因与邻居共享完整模型而仍存在表征能力不足的问题。为此，本文提出基于去中心化部分模型训练的个性化联邦学习框架DFedAlt。该框架通过对共享参数与个性化参数进行交替更新，以对等网络方式训练部分个性化的现代深度模型，从而精准个性化"正确"的模型组件。为促进共享参数聚合过程，我们进一步提出DFedSalt，其集成局部锐度感知最小化（SAM）优化器来更新共享参数。该优化器在梯度方向添加适当扰动，以克服客户端间共享模型不一致性问题。理论上，我们针对PFL中非凸场景下的去中心化部分模型训练，给出了两种算法的收敛性分析。在多种数据划分设置下的多个真实世界数据集实验表明：(i) 去中心化训练更适用于部分个性化方法，相比现有最优PFL基线取得最优精度；(ii) 引入适度扰动的共享参数使部分个性化联邦学习更适用于去中心化训练，其中DFedSalt展现出最具竞争力的性能。