Large language models (LLMs) strengthen instruction-following capability through instruction-finetuning (IFT) on supervised instruction/response data. However, widely used IFT datasets (e.g., Alpaca's 52k data) surprisingly contain many low-quality instances with incorrect or irrelevant responses, which are misleading and detrimental to IFT. In this paper, we propose a simple and effective data selection strategy that automatically identifies and filters out low-quality data using a strong LLM (e.g., ChatGPT). To this end, we introduce AlpaGasus, which is finetuned on only 9k high-quality data filtered from the 52k Alpaca data. AlpaGasus significantly outperforms the original Alpaca as evaluated by GPT-4 on multiple test sets and the controlled human evaluation. Its 13B variant matches $>90\%$ performance of its teacher LLM (i.e., Text-Davinci-003 generating the 52k data) on test tasks. It also provides 5.7x faster training, reducing the training time for a 7B variant from 80 minutes (for Alpaca) to 14 minutes. Moreover, the experiments prove the efficacy of our method across diverse datasets, base models, and LLM filters. Overall, AlpaGasus demonstrates a novel data-centric IFT paradigm that can be generally applied to instruction-tuning data, leading to faster training and better instruction-following models. Our project page is available at: https://lichang-chen.github.io/AlpaGasus/

翻译：大语言模型（LLMs）通过在有监督的指令/响应数据上进行指令微调（IFT）来增强指令遵循能力。然而，广泛使用的IFT数据集（例如Alpaca的52k数据）出人意料地包含许多低质量实例，其响应不正确或不相关，这对IFT具有误导性和危害性。在本文中，我们提出一种简单有效的数据选择策略，利用强大LLM（如ChatGPT）自动识别并过滤掉低质量数据。为此，我们引入AlpaGasus，它仅基于从52k Alpaca数据中筛选出的9k高质量数据进行微调。经GPT-4在多个测试集及受控人工评估上的评价，AlpaGasus显著优于原始Alpaca。其13B变体在测试任务上与教师LLM（即生成52k数据的Text-Davinci-003）性能匹配度超过90%。同时，它实现了5.7倍的训练加速，将7B变体的训练时间从80分钟（Alpaca）缩短至14分钟。此外，实验证明我们的方法在多样化数据集、基础模型和LLM过滤器上均有效。总体而言，AlpaGasus展示了一种以数据为中心的新型IFT范式，可普遍应用于指令微调数据，从而实现更快速训练和更优指令遵循模型。我们的项目页面位于：https://lichang-chen.github.io/AlpaGasus/