Personalized Federated Learning (PFL) is proposed to find the greatest personalized models for each client. To avoid the central failure and communication bottleneck in the server-based FL, we concentrate on the Decentralized Personalized Federated Learning (DPFL) that performs distributed model training in a Peer-to-Peer (P2P) manner. Most personalized works in DPFL are based on undirected and symmetric topologies, however, the data, computation and communication resources heterogeneity result in large variances in the personalized models, which lead the undirected aggregation to suboptimal personalized performance and unguaranteed convergence. To address these issues, we propose a directed collaboration DPFL framework by incorporating stochastic gradient push and partial model personalized, called \textbf{D}ecentralized \textbf{Fed}erated \textbf{P}artial \textbf{G}radient \textbf{P}ush (\textbf{DFedPGP}). It personalizes the linear classifier in the modern deep model to customize the local solution and learns a consensus representation in a fully decentralized manner. Clients only share gradients with a subset of neighbors based on the directed and asymmetric topologies, which guarantees flexible choices for resource efficiency and better convergence. Theoretically, we show that the proposed DFedPGP achieves a superior convergence rate of $\mathcal{O}(\frac{1}{\sqrt{T}})$ in the general non-convex setting, and prove the tighter connectivity among clients will speed up the convergence. The proposed method achieves state-of-the-art (SOTA) accuracy in both data and computation heterogeneity scenarios, demonstrating the efficiency of the directed collaboration and partial gradient push.

翻译：个性化联邦学习旨在为每个客户端寻找最优的个性化模型。为避免基于服务器的联邦学习中存在的中心故障与通信瓶颈，本研究聚焦于以点对点方式进行分布式模型训练的去中心化个性化联邦学习。现有去中心化个性化研究大多基于无向对称拓扑结构，然而数据、计算与通信资源的异构性会导致个性化模型产生较大方差，使得无向聚合策略难以获得最优个性化性能且无法保证收敛。为解决这些问题，本文提出一种融合随机梯度推送与部分模型个性化的定向协作去中心化联邦学习框架，称为去中心化联邦部分梯度推送。该框架通过个性化现代深度模型中的线性分类器以定制本地解决方案，并以完全去中心化的方式学习共识表征。客户端仅基于有向非对称拓扑与部分邻居共享梯度，这既保证了资源选择的灵活性以提升效率，也促进了更优的收敛性。理论分析表明，所提方法在一般非凸场景下达到$\mathcal{O}(\frac{1}{\sqrt{T}})$的优越收敛速率，并证明客户端间更紧密的连接将加速收敛。在数据异构与计算异构场景下，该方法均取得了最先进的准确率，验证了定向协作与部分梯度推送机制的有效性。