Instruction-tuning can be substantially optimized through enhanced diversity, resulting in models capable of handling a broader spectrum of tasks. However, existing data employed for such tuning often exhibit an inadequate coverage of individual domains, limiting the scope for nuanced comprehension and interactions within these areas. To address this deficiency, we propose Explore-Instruct, a novel approach to enhance the data coverage to be used in domain-specific instruction-tuning through active exploration via Large Language Models (LLMs). Built upon representative domain use cases, Explore-Instruct explores a multitude of variations or possibilities by implementing a search algorithm to obtain diversified and domain-focused instruction-tuning data. Our data-centric analysis validates the effectiveness of this proposed approach in improving domain-specific instruction coverage. Moreover, our model's performance demonstrates considerable advancements over multiple baselines, including those utilizing domain-specific data enhancement. Our findings offer a promising opportunity to improve instruction coverage, especially in domain-specific contexts, thereby advancing the development of adaptable language models. Our code, model weights, and data are public at \url{https://github.com/fanqiwan/Explore-Instruct}.

翻译：指令微调可以通过增强多样性实现显著优化，从而产生能够处理更广泛任务的模型。然而，现有用于此类微调的数据往往对个体领域的覆盖不足，限制了在这些领域内实现细致理解与交互的能力。为弥补这一缺陷，我们提出探索指令（Explore-Instruct）——一种通过大型语言模型的主动探索来增强领域特定指令微调数据覆盖范围的新方法。该方法基于代表性领域用例，通过实施搜索算法探索多种变体或可能性，从而获得多样化且聚焦领域的指令微调数据。我们的数据导向分析验证了该方法在提升领域特定指令覆盖方面的有效性。此外，我们的模型在多个基线（包括采用领域特定数据增强的方法）上展现出显著性能提升。我们的研究成果为改善指令覆盖（尤其在领域特定情境中）提供了可行路径，进而推动适应性语言模型的发展。我们的代码、模型权重及数据已公开于\url{https://github.com/fanqiwan/Explore-Instruct}。