Federated Continual Learning (FCL) has emerged as a promising paradigm that combines Federated Learning (FL) and Continual Learning (CL). To achieve good model accuracy, FCL needs to tackle catastrophic forgetting due to concept drift over time in CL, and to overcome the potential interference among clients in FL. We propose Concept Matching (CM), a clustering-based framework for FCL to address these challenges. The CM framework groups the client models into concept model clusters, and then builds different global models to capture different concepts in FL over time. In each round, the server sends the global concept models to the clients. To avoid catastrophic forgetting, each client selects the concept model best-matching the concept of the current data for further fine-tuning. To avoid interference among client models with different concepts, the server clusters the models representing the same concept, aggregates the model weights in each cluster, and updates the global concept model with the cluster model of the same concept. Since the server does not know the concepts captured by the aggregated cluster models, we propose a novel server concept matching algorithm that effectively updates a global concept model with a matching cluster model. The CM framework provides flexibility to use different clustering, aggregation, and concept matching algorithms. The evaluation demonstrates that CM outperforms state-of-the-art systems and scales well with the number of clients and the model size.

翻译：联邦持续学习（FCL）已成为一种结合联邦学习（FL）与持续学习（CL）的有前景的范式。为实现良好的模型精度，FCL需要应对CL中因概念漂移随时间推移导致的灾难性遗忘，并克服FL中客户端之间潜在的相互干扰。我们提出概念匹配（CM），一种基于聚类的FCL框架来解决这些挑战。CM框架将客户端模型分组为概念模型聚类，随后构建不同的全局模型以随时间捕捉FL中的不同概念。在每一轮中，服务器将全局概念模型发送至客户端。为避免灾难性遗忘，每个客户端选择与当前数据概念最匹配的概念模型进行进一步微调。为避免具有不同概念的客户端模型之间的相互干扰，服务器对代表相同概念的模型进行聚类，聚合每个聚类中的模型权重，并用同一概念的聚类模型更新全局概念模型。由于服务器不知道聚合后的聚类模型所捕捉的概念，我们提出一种新颖的服务器概念匹配算法，该算法能有效用匹配的聚类模型更新全局概念模型。CM框架提供了使用不同聚类、聚合及概念匹配算法的灵活性。评估表明，CM优于现有最先进系统，并能在客户端数量和模型规模方面良好扩展。