Personalized federated learning (PFL) is an approach proposed to address the issue of poor convergence on heterogeneous data. However, most existing PFL frameworks require strong assumptions for convergence. In this paper, we propose an alternating direction method of multipliers (ADMM) for training PFL models with Moreau envelope (FLAME), which achieves a sublinear convergence rate, relying on the relatively weak assumption of gradient Lipschitz continuity. Moreover, due to the gradient-free nature of ADMM, FLAME alleviates the need for hyperparameter tuning, particularly in avoiding the adjustment of the learning rate when training the global model. In addition, we propose a biased client selection strategy to expedite the convergence of training of PFL models. Our theoretical analysis establishes the global convergence under both unbiased and biased client selection strategies. Our experiments validate that FLAME, when trained on heterogeneous data, outperforms state-of-the-art methods in terms of model performance. Regarding communication efficiency, it exhibits an average speedup of 3.75x compared to the baselines. Furthermore, experimental results validate that the biased client selection strategy speeds up the convergence of both personalized and global models.

翻译：个性化联邦学习（PFL）是为解决异构数据上收敛性差问题而提出的方法。然而，现有大多数PFL框架需要强假设条件才能保证收敛。本文提出一种基于Moreau包络的交替方向乘子法（FLAME）来训练PFL模型，该方法在仅依赖梯度Lipschitz连续性这一较弱假设的条件下，实现了次线性收敛速率。此外，得益于ADMM的无梯度特性，FLAME减少了超参数调优需求，特别是在训练全局模型时避免了学习率的调整。同时，我们提出一种有偏客户端选择策略来加速PFL模型的训练收敛。理论分析证明了在无偏和有偏客户端选择策略下的全局收敛性。实验验证表明，FLAME在异构数据上训练时，模型性能优于现有最优方法。在通信效率方面，相比基线方法实现平均3.75倍的加速。此外，实验结果证实有偏客户端选择策略能同时加速个性化模型和全局模型的收敛。