In the current landscape of large language models (LLMs), the process of instruction tuning serves as an essential step. Considering the high computing power overhead, data-efficient instruction tuning was proposed to reduce the training data size in this process, aiming at selecting high-quality instructional data. Nevertheless, we argue that most current data-efficient instruction-tuning methods are highly dependent on the quality of the original instruction-tuning dataset. When it comes to datasets synthesized by LLMs, a common scenario in this field, dirty samples will even be selected with a higher probability than other samples. To address these challenges, we utilized external knowledge (relevant examples or paragraphs) to evaluate those samples synthesized by LLMs with an in-context-based relative predictive entropy. Based on the new metric, we proposed a framework, dubbed as \textbf{RECOST}, which integrates external-knowledge-base re-ranking and diversity-consistent sampling into a single pipeline. Through extensive experiments on several synthetic datasets (Alpaca and Alpaca-gpt4), we demonstrate the effectiveness of our method and achieve even better results with only \textbf{1\%} of the full dataset.

翻译：摘要：在当前大型语言模型（LLMs）的发展格局中，指令微调过程是一个关键步骤。考虑到高昂的计算开销，数据高效指令微调被提出以降低该过程中的训练数据规模，旨在筛选出高质量的指令数据。然而，我们认为当前大多数数据高效指令微调方法高度依赖于原始指令微调数据集的质量。当面对由LLMs合成（该领域常见情况）的数据集时，脏样本甚至会比其它样本有更高概率被选中。为解决这些挑战，我们利用外部知识（相关示例或段落），基于上下文相关的相对预测熵来评估由LLMs合成的样本。基于这一新指标，我们提出了一个名为\textbf{RECOST}的框架，该框架将外部知识库重排序与多样性一致性采样整合到单一流程中。通过在多个合成数据集（Alpaca和Alpaca-gpt4）上进行大量实验，我们证明了该方法的有效性，并且仅使用完整数据集的\textbf{1\%}就取得了更优的结果。