Federated continual learning (FCL) learns incremental tasks over time from confidential datasets distributed across clients. This paper focuses on rehearsal-free FCL, which has severe forgetting issues when learning new tasks due to the lack of access to historical task data. To address this issue, we propose Fed-CPrompt based on prompt learning techniques to obtain task-specific prompts in a communication-efficient way. Fed-CPrompt introduces two key components, asynchronous prompt learning, and contrastive continual loss, to handle asynchronous task arrival and heterogeneous data distributions in FCL, respectively. Extensive experiments demonstrate the effectiveness of Fed-CPrompt in achieving SOTA rehearsal-free FCL performance.
翻译:[译摘要] 联邦持续学习(FCL)需要在保护客户端机密数据集的前提下,随时间推移学习不断新增的任务。本文聚焦于无重演联邦持续学习场景,该类场景因无法访问历史任务数据,在学习新任务时存在严重的遗忘问题。为此,我们提出基于提示学习技术的Fed-CPrompt方法,以通信高效的方式获取任务专用提示。Fed-CPrompt引入两个关键组件:异步提示学习与对比持续损失函数,分别处理联邦持续学习中的任务异步到达与数据异构分布问题。大量实验证明,Fed-CPrompt在无重演联邦持续学习任务上取得了当前最优(SOTA)性能。