Instruction tuning has become the de facto method to equip large language models (LLMs) with the ability of following user instructions. Usually, hundreds of thousands or millions of instruction-following pairs are employed to fine-tune the foundation LLMs. Recently, some studies show that a small number of high-quality instruction data is enough. However, how to select appropriate instruction data for a given LLM is still an open problem. To address this problem, in this paper we present a model-oriented data selection (MoDS) approach, which selects instruction data based on a new criteria considering three aspects: quality, coverage and necessity. First, our approach utilizes a quality evaluation model to filter out the high-quality subset from the original instruction dataset, and then designs an algorithm to further select from the high-quality subset a seed instruction dataset with good coverage. The seed dataset is applied to fine-tune the foundation LLM to obtain an initial instruction-following LLM. Finally, we develop a necessity evaluation model to find out the instruction data which are performed badly in the initial instruction-following LLM and consider them necessary instructions to further improve the LLMs. In this way, we can get a small high-quality, broad-coverage and high-necessity subset from the original instruction datasets. Experimental results show that, the model fine-tuned with 4,000 instruction pairs selected by our approach could perform better than the model fine-tuned with the full original dataset which includes 214k instruction data.

翻译：指令微调已成为赋予大型语言模型遵循用户指令能力的事实标准方法。通常采用数十万乃至数百万条指令跟随对来微调基础语言模型。近期研究表明，少量高质量指令数据即可达到预期效果，但如何针对特定语言模型选择恰当的指令数据仍是一个开放性问题。为解决该问题，本文提出一种面向模型的数据选择方法，该方法基于质量、覆盖度和必要性三个维度的新标准选取指令数据。首先，利用质量评估模型从原始指令数据集中筛选高质量子集，随后设计算法从该子集中进一步选取具有良好覆盖度的种子指令数据集。使用该种子数据集微调基础语言模型以获取初始指令跟随模型。最后，开发必要性评估模型识别初始指令跟随模型中表现欠佳的指令数据，将其视为提升模型性能的必要指令数据。通过这种流程，我们从原始指令数据集中获得包含高质量、广覆盖度和高必要性的精简子集。实验结果表明，采用本方法选取的4000条指令对微调的模型，其性能优于使用包含214k条指令数据的完整原始数据集微调的模型。