Instructions augmentation is a crucial step for unleashing the full potential of large language models (LLMs) in downstream tasks. Existing Self-Instruct methods primarily simulate new instructions from a few initial instructions with in-context learning. However, our study identifies a critical flaw in this approach: even with GPT4o, Self-Instruct cannot generate complex instructions of length $\ge 100$, which is necessary in complex tasks such as code completion. To address this issue, our key insight is that fine-tuning open source LLMs with only ten examples can produce complex instructions that maintain distributional consistency for complex reasoning tasks. We introduce Ada-Instruct, an adaptive instruction generator developed through fine-tuning. We empirically validated Ada-Instruct's efficacy across different applications. The results highlight Ada-Instruct's capacity to generate long, intricate, and distributionally consistent instructions.

翻译：指令增强是释放大型语言模型在下游任务中全部潜力的关键步骤。现有的自指令方法主要通过上下文学习从少量初始指令中模拟新指令。然而，我们的研究发现该方法存在一个关键缺陷：即使使用GPT4o，自指令方法也无法生成长度大于等于100的复杂指令，而这在代码补全等复杂任务中是必需的。为解决此问题，我们的核心见解是，仅使用十个示例对开源大型语言模型进行微调，即可生成在复杂推理任务中保持分布一致性的复杂指令。我们提出了Ada-Instruct，一种通过微调开发的自适应指令生成器。我们在不同应用中实证验证了Ada-Instruct的有效性。结果突显了Ada-Instruct生成长、复杂且分布一致指令的能力。