Instruction tuning for large language models (LLMs) can drive them to produce results consistent with human goals in specific downstream tasks. However, the process of continual instruction tuning (CIT) for LLMs may bring about the catastrophic forgetting (CF) problem, where previously learned abilities are degraded. Recent methods try to alleviate the CF problem by modifying models or replaying data, which may only remember the surface-level pattern of instructions and get confused on held-out tasks. In this paper, we propose a novel continual instruction tuning method based on Key-part Information Gain (KPIG). Our method computes the information gain on masked parts to dynamically replay data and refine the training objective, which enables LLMs to capture task-aware information relevant to the correct response and alleviate overfitting to general descriptions in instructions. In addition, we propose two metrics, P-score and V-score, to measure the generalization and instruction-following abilities of LLMs. Experiments demonstrate our method achieves superior performance on both seen and held-out tasks.

翻译：大型语言模型（LLM）的指令微调可驱动其在特定下游任务中生成与人类目标一致的结果。然而，对LLM进行持续指令微调（CIT）可能引发灾难性遗忘（CF）问题，导致先前习得的能力退化。现有方法试图通过修改模型或重放数据来缓解CF问题，但这可能仅记住指令的表面模式，在保留任务中产生混淆。本文提出一种基于关键部分信息增益（KPIG）的新型持续指令微调方法。该方法通过计算掩码部分的信息增益来动态重放数据并优化训练目标，使LLM能够捕获与正确响应相关的任务感知信息，同时缓解对指令中通用描述的过拟合。此外，我们提出两个评估指标——P-score与V-score——用于衡量LLM的泛化能力和指令遵循能力。实验表明，我们的方法在已见任务与保留任务中均实现了优越性能。