Instruction tuning has emerged as a promising approach to enhancing large language models in following human instructions. It is shown that increasing the diversity and number of instructions in the training data can consistently enhance generalization performance, which facilitates a recent endeavor to collect various instructions and integrate existing instruction tuning datasets into larger collections. However, different users have their unique ways of expressing instructions, and there often exist variations across different datasets in the instruction styles and formats, i.e., format inconsistency. In this work, we study how format inconsistency may impact the performance of instruction tuning. We propose a framework called "Unified Instruction Tuning" (UIT), which calls OpenAI APIs for automatic format transfer among different instruction tuning datasets. We show that UIT successfully improves the generalization performance on unseen instructions, which highlights the importance of format consistency for instruction tuning. To make the UIT framework more practical, we further propose a novel perplexity-based denoising method to reduce the noise of automatic format transfer. We also train a smaller offline model that achieves comparable format transfer capability than OpenAI APIs to reduce costs in practice.

翻译：指令微调已成为增强大语言模型遵循人类指令能力的一种有前景方法。研究表明，增加训练数据中指令的多样性和数量可以持续提升泛化性能，这推动了近期收集各类指令并将现有指令微调数据集整合为更大规模集合的研究工作。然而，不同用户表达指令的方式存在差异，且不同数据集在指令风格和格式上常存在差异，即格式不一致性。本文研究格式不一致性可能对指令微调性能产生的影响。我们提出名为"统一指令微调"（UIT）的框架，通过调用OpenAI API实现不同指令微调数据集间的自动格式转换。实验表明，UIT成功提升了模型在未见指令上的泛化性能，凸显了格式一致性对指令微调的重要性。为使UIT框架更具实用性，我们进一步提出基于困惑度的新颖去噪方法，以降低自动格式转换的噪声。同时，我们训练了一个更小规模的离线模型，其格式转换能力可与OpenAI API相媲美，从而降低实际应用成本。