Continual learning empowers models to adapt autonomously to the ever-changing environment or data streams without forgetting old knowledge. Prompt-based approaches are built on frozen pre-trained models to learn the task-specific prompts and classifiers efficiently. Existing prompt-based methods are inconsistent between training and testing, limiting their effectiveness. Two types of inconsistency are revealed. Test predictions are made from all classifiers while training only focuses on the current task classifier without holistic alignment, leading to Classifier inconsistency. Prompt inconsistency indicates that the prompt selected during testing may not correspond to the one associated with this task during training. In this paper, we propose a novel prompt-based method, Consistent Prompting (CPrompt), for more aligned training and testing. Specifically, all existing classifiers are exposed to prompt training, resulting in classifier consistency learning. In addition, prompt consistency learning is proposed to enhance prediction robustness and boost prompt selection accuracy. Our Consistent Prompting surpasses its prompt-based counterparts and achieves state-of-the-art performance on multiple continual learning benchmarks. Detailed analysis shows that improvements come from more consistent training and testing.

翻译：持续学习使模型能够自主适应不断变化的环境或数据流，而不会遗忘旧知识。基于提示的方法建立在冻结的预训练模型上，高效学习任务特定提示和分类器。现有基于提示的方法在训练和测试之间存在不一致性，限制了其有效性。揭示了两种不一致性：测试预测基于所有分类器进行，而训练仅关注当前任务分类器且缺乏全局对齐，导致分类器不一致性；提示不一致性则体现为测试期间选中的提示可能与训练中该任务对应的提示不一致。本文提出一种新颖的基于提示的方法——一致提示（CPrompt），以实现更对齐的训练与测试。具体而言，所有现有分类器均参与提示训练，从而实现分类器一致性学习。此外，提出提示一致性学习以增强预测鲁棒性并提高提示选择准确性。我们的一致提示方法优于同类基于提示的方法，并在多个持续学习基准上达到最先进性能。详细分析表明，性能提升源于更一致的训练与测试流程。