Continual learning empowers models to adapt autonomously to the ever-changing environment or data streams without forgetting old knowledge. Prompt-based approaches are built on frozen pre-trained models to learn the task-specific prompts and classifiers efficiently. Existing prompt-based methods are inconsistent between training and testing, limiting their effectiveness. Two types of inconsistency are revealed. Test predictions are made from all classifiers while training only focuses on the current task classifier without holistic alignment, leading to Classifier inconsistency. Prompt inconsistency indicates that the prompt selected during testing may not correspond to the one associated with this task during training. In this paper, we propose a novel prompt-based method, Consistent Prompting (CPrompt), for more aligned training and testing. Specifically, all existing classifiers are exposed to prompt training, resulting in classifier consistency learning. In addition, prompt consistency learning is proposed to enhance prediction robustness and boost prompt selection accuracy. Our Consistent Prompting surpasses its prompt-based counterparts and achieves state-of-the-art performance on multiple continual learning benchmarks. Detailed analysis shows that improvements come from more consistent training and testing.

翻译：持续学习使模型能够自主适应不断变化的环境或数据流，同时避免遗忘旧知识。基于提示的方法依托冻结的预训练模型，高效学习任务特定提示和分类器。现有基于提示的方法在训练与测试之间存在不一致性，限制了其有效性。本文揭示了两种不一致性：分类器不一致性指测试阶段预测基于所有分类器，而训练阶段仅聚焦当前任务分类器，缺乏全局对齐；提示不一致性指测试阶段所选提示可能与训练阶段该任务关联的提示不匹配。为此，我们提出一种新颖的基于提示的方法——一致提示法（CPrompt），旨在实现更对齐的训练与测试流程。具体而言，通过将所有现有分类器暴露于提示训练中，实现分类器一致性学习；同时引入提示一致性学习，以增强预测鲁棒性并提升提示选择准确性。我们的一致提示法超越了其他基于提示的方法，在多个持续学习基准上取得了最先进性能。详细分析表明，性能提升源于更一致的训练与测试过程。