We focus on the problem of Personalized Federated Continual Learning (PFCL): a group of distributed clients, each with a sequence of local tasks on arbitrary data distributions, collaborate through a central server to train a personalized model at each client, with the model expected to achieve good performance on all local tasks. We propose a novel PFCL framework called Federated Memory Strengthening FedMeS to address the challenges of client drift and catastrophic forgetting. In FedMeS, each client stores samples from previous tasks using a small amount of local memory, and leverages this information to both 1) calibrate gradient updates in training process; and 2) perform KNN-based Gaussian inference to facilitate personalization. FedMeS is designed to be task-oblivious, such that the same inference process is applied to samples from all tasks to achieve good performance. FedMeS is analyzed theoretically and evaluated experimentally. It is shown to outperform all baselines in average accuracy and forgetting rate, over various combinations of datasets, task distributions, and client numbers.

翻译：我们聚焦于个性化联邦持续学习（PFCL）问题：一组分布式客户端，每个客户端在任意数据分布上拥有一系列本地任务，通过中央服务器协作，为每个客户端训练一个个性化模型，且该模型需在所有本地任务上实现良好性能。我们提出一种名为“联邦记忆增强”（FedMeS）的新型PFCL框架，以应对客户端漂移和灾难性遗忘的挑战。在FedMeS中，每个客户端利用少量本地内存存储先前任务的样本，并利用这些信息实现：1）在训练过程中校正梯度更新；2）执行基于KNN的高斯推理以促进个性化。FedMeS被设计为任务无关的，即对所有任务的样本应用相同的推理过程以实现优异性能。我们对FedMeS进行了理论分析与实验评估。结果表明，在不同数据集、任务分布和客户端数量的组合下，FedMeS在平均准确率和遗忘率上均优于所有基线方法。