Continual learning remains a fundamental challenge in machine learning, requiring models to learn from a stream of tasks without forgetting previously acquired knowledge. A major obstacle in this setting is catastrophic forgetting, where performance on earlier tasks degrades as new tasks are learned. In this paper, we introduce PPSEBM, a novel framework that integrates an Energy-Based Model (EBM) with Progressive Parameter Selection (PPS) to effectively address catastrophic forgetting in continual learning for natural language processing tasks. In PPSEBM, progressive parameter selection allocates distinct, task-specific parameters for each new task, while the EBM generates representative pseudo-samples from prior tasks. These generated samples actively inform and guide the parameter selection process, enhancing the model's ability to retain past knowledge while adapting to new tasks. Experimental results on diverse NLP benchmarks demonstrate that PPSEBM outperforms state-of-the-art continual learning methods, offering a promising and robust solution to mitigate catastrophic forgetting.

翻译：持续学习仍然是机器学习中的一个根本性挑战，要求模型能够从一系列任务中持续学习，同时不遗忘先前已掌握的知识。在此背景下，一个主要障碍是灾难性遗忘，即随着新任务的学习，模型在早期任务上的性能会显著下降。本文提出PPSEBM，一种新颖的框架，它将基于能量的模型与渐进参数选择相结合，以有效应对自然语言处理任务中持续学习所面临的灾难性遗忘问题。在PPSEBM中，渐进参数选择为每个新任务分配独特且任务特定的参数，而EBM则从先前任务中生成具有代表性的伪样本。这些生成的样本主动指导并优化参数选择过程，从而增强模型在适应新任务的同时保留过去知识的能力。在多种自然语言处理基准测试上的实验结果表明，PPSEBM优于当前最先进的持续学习方法，为缓解灾难性遗忘提供了一个前景广阔且稳健的解决方案。