The surge in Large Language Models (LLMs) has revolutionized natural language processing, but fine-tuning them for specific tasks often encounters challenges in balancing performance and preserving general instruction-following abilities. In this paper, we posit that the distribution gap between task datasets and the LLMs serves as the primary underlying cause. To address the problem, we introduce Self-Distillation Fine-Tuning (SDFT), a novel approach that bridges the distribution gap by guiding fine-tuning with a distilled dataset generated by the model itself to match its original distribution. Experimental results on the Llama-2-chat model across various benchmarks demonstrate that SDFT effectively mitigates catastrophic forgetting while achieving comparable or superior performance on downstream tasks compared to the vanilla fine-tuning. Moreover, SDFT demonstrates the potential to maintain the helpfulness and safety alignment of LLMs. Our code is available at \url{https://github.com/sail-sg/sdft}.

翻译：大规模语言模型（LLMs）的兴起彻底革新了自然语言处理，但针对特定任务进行微调时，通常会在平衡性能与保持通用指令遵循能力方面遇到挑战。本文提出，任务数据集与LLMs之间的分布差距是导致这一问题的根本原因。为应对该问题，我们引入自我蒸馏微调（SDFT），这是一种通过利用模型自身生成的蒸馏数据集来引导微调过程、使其与原始分布相匹配的新方法，从而弥合分布差距。在Llama-2-chat模型上跨多个基准的实验结果表明，SDFT在有效缓解灾难性遗忘的同时，在下游任务上取得了与标准微调相当或更优的性能。此外，SDFT展现出保持LLMs有用性与安全性对齐的潜力。我们的代码已开源在 \url{https://github.com/sail-sg/sdft}。