Continual learning (CL) is a learning paradigm that emulates the human capability of learning and accumulating knowledge continually without forgetting the previously learned knowledge and also transferring the learned knowledge to help learn new tasks better. This survey presents a comprehensive review and analysis of the recent progress of CL in NLP, which has significant differences from CL in computer vision and machine learning. It covers (1) all CL settings with a taxonomy of existing techniques; (2) catastrophic forgetting (CF) prevention, (3) knowledge transfer (KT), which is particularly important for NLP tasks; and (4) some theory and the hidden challenge of inter-task class separation (ICS). (1), (3) and (4) have not been included in the existing survey. Finally, a list of future directions is discussed.

翻译：持续学习是一种学习范式，它模拟人类持续学习和积累知识的能力，既能不遗忘先前学到的知识，又能将所学知识迁移以更好地帮助学习新任务。本综述全面回顾并分析了自然语言处理（NLP）中持续学习的最新进展——其与计算机视觉及机器学习中的持续学习存在显著差异。内容涵盖：(1) 所有持续学习设置及现有技术的分类体系；(2) 灾难性遗忘(CF)预防；(3) 对NLP任务尤为关键的知识迁移(KT)；(4) 相关理论及任务间类别分离(ICS)这一隐性挑战。其中(1)、(3)、(4)在现有综述中尚未被涵盖。最后，本文讨论了若干未来研究方向。