Large language models (LLMs) suffer from catastrophic forgetting during continual learning. Conventional rehearsal-based methods rely on previous training data to retain the model's ability, which may not be feasible in real-world applications. When conducting continual learning based on a publicly-released LLM checkpoint, the availability of the original training data may be non-existent. To address this challenge, we propose a framework called Self-Synthesized Rehearsal (SSR) that uses the LLM to generate synthetic instances for rehearsal. Concretely, we first employ the base LLM for in-context learning to generate synthetic instances. Subsequently, we utilize the latest LLM to refine the instance outputs based on the synthetic inputs, preserving its acquired ability. Finally, we select diverse high-quality synthetic instances for rehearsal in future stages. Experimental results demonstrate that SSR achieves superior or comparable performance compared to conventional rehearsal-based approaches while being more data-efficient. Besides, SSR effectively preserves the generalization capabilities of LLMs in general domains.

翻译：大型语言模型在持续学习过程中面临灾难性遗忘的问题。传统的基于回放的方法依赖于保留模型能力的先前训练数据，这在现实应用中可能不可行。当基于公开发布的大型语言模型检查点进行持续学习时，原始训练数据的可用性可能不存在。为解决这一挑战，我们提出了一种名为"自合成回放"（SSR）的框架，该框架利用大型语言模型生成用于回放的合成实例。具体来说，我们首先利用基础大型语言模型进行上下文学习以生成合成实例，随后使用最新的大型语言模型基于合成输入优化实例输出，从而保留其已获取的能力。最后，我们选择多样化的高质量合成实例用于后续阶段的回放。实验结果表明，SSR在保持更高效数据利用率的同时，达到了与基于传统回放方法相当或更优的性能。此外，SSR有效保留了大型语言模型在通用领域的泛化能力。