Fine-tuning large language models (LLMs) is essential for enhancing their performance on specific tasks but is often resource-intensive due to redundant or uninformative data. To address this inefficiency, we introduce DELIFT (Data Efficient Language model Instruction Fine-Tuning), a novel algorithm that systematically optimizes data selection across the three key stages of fine-tuning: (1) instruction tuning, (2) task-specific fine-tuning (e.g., reasoning, question-answering), and (3) continual fine-tuning (e.g., incorporating new data versions). Unlike existing methods that focus on single-stage optimization or rely on computationally intensive gradient calculations, DELIFT operates efficiently across all stages. Central to our approach is a pairwise utility metric that quantifies how beneficial a data sample is for improving the model's responses to other samples, effectively measuring the informational value relative to the model's current capabilities. By leveraging different submodular functions applied to this metric, DELIFT selects diverse and optimal subsets that are useful across all stages of fine-tuning. Experiments across various tasks and model scales demonstrate that DELIFT can reduce the fine-tuning data size by up to 70% without compromising performance, offering significant computational savings and outperforming existing methods in both efficiency and efficacy.

翻译：微调大型语言模型（LLM）对于提升其在特定任务上的性能至关重要，但由于数据冗余或信息量不足，该过程通常资源密集。为解决这一效率问题，我们提出了DELIFT（数据高效的语言模型指令微调），这是一种新颖的算法，它系统性地优化了微调三个关键阶段的数据选择：（1）指令微调，（2）任务特定微调（例如推理、问答），以及（3）持续微调（例如纳入新版本数据）。与现有方法通常专注于单阶段优化或依赖计算密集的梯度计算不同，DELIFT能高效地跨所有阶段运行。我们方法的核心是一个成对效用度量，它量化了一个数据样本对于改进模型对其他样本的响应有多大益处，从而有效衡量了相对于模型当前能力的信息价值。通过利用应用于此度量的不同子模函数，DELIFT选择了多样化且最优的数据子集，这些子集在微调的所有阶段都很有用。在各种任务和模型规模上的实验表明，DELIFT能将微调数据量减少高达70%而不影响性能，提供了显著的计算节省，并且在效率和效果上都优于现有方法。