Instruction-tuning can be substantially optimized through enhanced diversity, resulting in models capable of handling a broader spectrum of tasks. However, existing data employed for such tuning often exhibit an inadequate coverage of individual domains, limiting the scope for nuanced comprehension and interactions within these areas. To address this deficiency, we propose Explore-Instruct, a novel approach to enhance the data coverage to be used in domain-specific instruction-tuning through active exploration via Large Language Models (LLMs). Built upon representative domain use cases, Explore-Instruct explores a multitude of variations or possibilities by implementing a search algorithm to obtain diversified and domain-focused instruction-tuning data. Our data-centric analysis validates the effectiveness of this proposed approach in improving domain-specific instruction coverage. Moreover, our model's performance demonstrates considerable advancements over multiple baselines, including those utilizing domain-specific data enhancement. Our findings offer a promising opportunity to improve instruction coverage, especially in domain-specific contexts, thereby advancing the development of adaptable language models. Our code, model weights, and data are public at \url{https://github.com/fanqiwan/Explore-Instruct}.

翻译：指令微调可以通过增强多样性得到显著优化，从而培养出能处理更广泛任务的模型。然而，现有用于此类微调的数据往往在单个领域的覆盖上存在不足，限制了模型对这些领域的细致理解与交互能力。为解决这一缺陷，我们提出探索指令（Explore-Instruct），一种通过大型语言模型进行主动探索来增强领域特定指令微调数据覆盖范围的新方法。该方法基于代表性领域用例，通过实施搜索算法探索多种变化或可能性，获取多样化且聚焦于领域的指令微调数据。我们的数据驱动分析验证了该方法在提升领域特定指令覆盖范围方面的有效性。此外，我们的模型性能在多个基准（包括使用领域特定数据增强的方法）上展现出显著优势。我们的研究结果为提升指令覆盖范围（尤其在领域特定情境下）提供了有前景的机会，从而推动适应性语言模型的发展。我们的代码、模型权重和数据已在 \url{https://github.com/fanqiwan/Explore-Instruct} 公开。