Continual learning refers to the capability of continuously learning from a stream of data. Current research mainly focuses on relieving catastrophic forgetting, and most of their success is at the cost of limiting the performance of newly incoming tasks. Such a trade-off is referred to as the stabilityplasticity dilemma and is a more general and challenging problem for continual learning. However, the inherent conflict between these two concepts makes it seemingly impossible to devise a satisfactory solution to both of them simultaneously. Therefore, we ask, "is it possible to divide them into two problems to conquer independently?" To this end, we propose a prompt-tuning-based method termed PromptFusion to enable the decoupling of stability and plasticity. Specifically, PromptFusion consists of a carefully designed Stabilizer module that deals with catastrophic forgetting and a Booster module to learn new knowledge concurrently. During training, PromptFusion first passes an input image to the two modules separately. Then the resulting logits are further fused with a learnable weight parameter. Finally, a weight mask is applied to the derived logits to balance between old and new classes. Extensive experiments show that our method achieves promising results on popular continual learning datasets for both class-incremental and domain incremental settings. Especially on Split-Imagenet-R, one of the most challenging datasets for class-incremental learning, our method exceeds state-of-the-art prompt-based methods L2P and DualPrompt by more than 10%.

翻译：持续学习是指从数据流中持续学习的能力。当前研究主要聚焦于缓解灾难性遗忘，但多数成果以限制新任务性能为代价。这种权衡被称为稳定性-可塑性困境，是持续学习中更普遍且更具挑战性的问题。然而，这两个概念的内在冲突使得同时获得两者的理想解看似不可能。因此我们提出："是否可能将这两个问题分而治之？"为此，我们提出基于提示调优的方法PromptFusion，实现稳定性与可塑性的解耦。具体而言，PromptFusion包含精心设计的用于处理灾难性遗忘的稳定器模块和用于并行学习新知识的增强器模块。训练时，PromptFusion先将输入图像分别传入两个模块，再通过可学习权重参数融合所得logits，最后对导出logits施加权重掩码以平衡新旧类别。大量实验表明，该方法在类增量与域增量两种场景的经典持续学习数据集上均取得优异性能。特别是在类增量学习最具挑战性的数据集Split-Imagenet-R上，我们的方法比当前最先进的基于提示的方法L2P和DualPrompt性能提升超过10%。