While instructions fine-tuning of large language models (LLMs) has been proven to enhance performance across various applications, the influence of the instruction dataset mixture on LLMs has not been thoroughly explored. In this study, we classify instructions into three main types: NLP downstream tasks, coding, and general chatting, and investigate their impact on LLMs. Our findings reveal that specific types of instructions are more beneficial for particular uses, while it may cause harms to other aspects, emphasizing the importance of meticulously designing the instruction mixture to maximize model performance. This study sheds light on the instruction mixture and paves the way for future research.

翻译：虽然指令微调已证明能提升大语言模型在各种应用中的性能，但指令数据集混合对模型的影响尚未得到充分探索。在本研究中，我们将指令分为三大类型：自然语言处理下游任务、编程和通用对话，并探究它们对大语言模型的影响。我们的发现表明，特定类型的指令对某些用途更有益，但可能对其他方面造成损害，这强调了精心设计指令混合以最大化模型性能的重要性。本研究阐明了指令混合机制，并为未来研究奠定了基础。