The ability of large language models (LLMs) to follow instructions is crucial to real-world applications. Despite recent advances, several studies have highlighted that LLMs struggle when faced with challenging instructions, especially those that include complex constraints, hindering their effectiveness in various tasks. To address this challenge, we introduce Conifer, a novel instruction tuning dataset, designed to enhance LLMs to follow multi-level instructions with complex constraints. Utilizing GPT-4, we curate the dataset by a series of LLM-driven refinement processes to ensure high quality. We also propose a progressive learning scheme that emphasizes an easy-to-hard progression, and learning from process feedback. Models trained with Conifer exhibit remarkable improvements in instruction-following abilities, especially for instructions with complex constraints. On several instruction-following benchmarks, our 7B model outperforms the state-of-the-art open-source 7B models, even exceeds the performance of models 10 times larger on certain metrics. All the code and Conifer dataset are available at https://www.github.com/ConiferLM/Conifer.

翻译：大语言模型遵循指令的能力对于实际应用至关重要。尽管近期取得了进展，但多项研究表明，大语言模型在面对具有挑战性的指令（尤其是包含复杂约束的指令）时表现欠佳，这阻碍了其在各类任务中的有效性。为解决这一挑战，我们提出了Conifer——一种新型指令调优数据集，旨在增强大语言模型遵循包含复杂约束的多层级指令的能力。利用GPT-4，我们通过一系列由大语言模型驱动的精炼流程来构建该数据集，以确保其高质量。我们还提出了一种渐进式学习方案，强调由易到难的递进过程以及从过程反馈中学习。经Conifer训练的模型在指令遵循能力上展现出显著提升，尤其对于包含复杂约束的指令。在多项指令遵循基准测试中，我们的7B模型超越了最先进的开源7B模型，甚至在特定指标上超过了规模大10倍的模型性能。所有代码及Conifer数据集均可在https://www.github.com/ConiferLM/Conifer获取。