Instruction Tuning (IT), the process of training large language models (LLMs) using instruction-response pairs, has emerged as the predominant method for transforming base pre-trained LLMs into open-domain conversational agents. While IT has achieved notable success and widespread adoption, its limitations and shortcomings remain underexplored. In this paper, through rigorous experiments and an in-depth analysis of the changes LLMs undergo through IT, we reveal various limitations of IT. In particular, we show that (1) IT fails to enhance knowledge or skills in LLMs. LoRA fine-tuning is limited to learning response initiation and style tokens, and full-parameter fine-tuning leads to knowledge degradation. (2) Copying response patterns from IT datasets derived from knowledgeable sources leads to a decline in response quality. (3) Full-parameter fine-tuning increases hallucination by inaccurately borrowing tokens from conceptually similar instances in the IT dataset for generating responses. (4) Popular methods to improve IT do not lead to performance improvements over a simple LoRA fine-tuned model. Our findings reveal that responses generated solely from pre-trained knowledge consistently outperform responses by models that learn any form of new knowledge from IT on open-source datasets. We hope the insights and challenges revealed inspire future work.

翻译：指令微调（Instruction Tuning，IT）是指使用指令-响应对训练大型语言模型（LLMs）的过程，已成为将预训练基础LLMs转化为开放域对话智能体的主流方法。尽管IT取得了显著成功并得到广泛应用，但其局限性和缺陷仍未被充分探索。本文通过严谨的实验和深入分析LLMs在IT过程中发生的变化，揭示了IT的多种局限性。具体而言，我们发现：（1）IT未能增强LLMs的知识或技能。LoRA微调仅限于学习响应启动和风格标记，而全参数微调会导致知识退化。（2）从知识来源丰富的IT数据集中复制响应模式会导致响应质量下降。（3）全参数微调通过不准确地从IT数据集中语义相似的实例中借用标记来生成响应，从而加剧了幻觉现象。（4）改进IT的流行方法并未在性能上超越简单的LoRA微调模型。我们的研究表明，仅基于预训练知识生成的响应始终优于那些通过IT从开源数据集中学习任何形式新知识的模型。我们希望所揭示的见解与挑战能够启发未来研究。