We study the personalized federated learning problem under asynchronous updates. In this problem, each client seeks to obtain a personalized model that simultaneously outperforms local and global models. We consider two optimization-based frameworks for personalization: (i) Model-Agnostic Meta-Learning (MAML) and (ii) Moreau Envelope (ME). MAML involves learning a joint model adapted for each client through fine-tuning, whereas ME requires a bi-level optimization problem with implicit gradients to enforce personalization via regularized losses. We focus on improving the scalability of personalized federated learning by removing the synchronous communication assumption. Moreover, we extend the studied function class by removing boundedness assumptions on the gradient norm. Our main technical contribution is a unified proof for asynchronous federated learning with bounded staleness that we apply to MAML and ME personalization frameworks. For the smooth and non-convex functions class, we show the convergence of our method to a first-order stationary point. We illustrate the performance of our method and its tolerance to staleness through experiments for classification tasks over heterogeneous datasets.

翻译：我们研究异步更新下的个性化联邦学习问题。在该问题中，每个客户端旨在获得一个同时优于本地模型和全局模型的个性化模型。我们考虑两种基于优化的个性化框架：（i）模型无关元学习（MAML）和（ii）Moreau包络（ME）。MAML通过学习一个可通过微调适应每个客户端的联合模型，而ME则要求通过隐式梯度求解双层优化问题，借助正则化损失实现个性化。我们通过移除同步通信假设，专注于提升个性化联邦学习的可扩展性。此外，我们通过移除梯度范数的有界性假设，扩展了所研究的函数类。我们的主要技术贡献是一个适用于MAML和ME个性化框架的、针对有界延迟异步联邦学习的统一证明。针对光滑非凸函数类，我们证明了所提方法收敛至一阶驻点。通过异构数据集上的分类任务实验，我们展示了所提方法的性能及其对延迟的容忍性。