As world knowledge evolves and new task paradigms emerge, Continual Learning (CL) is crucial for keeping Large Language Models (LLMs) up-to-date and addressing their shortcomings. In practical applications, LLMs often require both continual instruction tuning (CIT) and continual pre-training (CPT) to adapt to new task paradigms and acquire necessary knowledge for task-solving. However, it remains challenging to collect CPT data that addresses the knowledge deficiencies in models while maintaining adequate volume, and improving the efficiency of utilizing this data also presents significant difficulties. Inspired by the 'summarizing mistakes' learning skill, we propose the Continue Evolving from Mistakes (CEM) method, aiming to provide a data-efficient approach for collecting CPT data and continually improving LLMs' performance through iterative evaluation and supplementation with mistake-relevant knowledge. To efficiently utilize these CPT data and mitigate forgetting, we design a novel CL training set construction paradigm that integrates parallel CIT and CPT data. Extensive experiments demonstrate the efficacy of the CEM method, achieving up to a 17% improvement in accuracy in the best case. Furthermore, additional experiments confirm the potential of combining CEM with catastrophic forgetting mitigation methods, enabling iterative and continual model evolution.

翻译：随着世界知识的演进与新任务范式的出现，持续学习对于保持大型语言模型（LLMs）的时效性及弥补其缺陷至关重要。在实际应用中，LLMs通常需要同时进行持续指令微调（CIT）与持续预训练（CPT），以适应新的任务范式并获取任务解决所需的知识。然而，如何收集既能针对模型知识缺陷、又能保持足够规模的CPT数据仍具挑战，且提升此类数据的利用效率亦存在显著困难。受“总结错误”学习技能的启发，我们提出“从错误中持续演化”方法，旨在通过迭代评估与补充错误相关知识，为CPT数据收集提供一种高效的数据利用途径，并持续提升LLMs性能。为高效利用这些CPT数据并缓解遗忘，我们设计了一种融合并行CIT与CPT数据的新型持续学习训练集构建范式。大量实验证明了CEM方法的有效性，在最佳情况下实现了高达17%的准确率提升。此外，补充实验证实了CEM与灾难性遗忘缓解方法结合的可能性，从而支持模型的迭代式持续演化。