Question answering represents a core capability of large language models (LLMs). However, when individuals encounter unfamiliar knowledge in texts, they often formulate questions that the text itself cannot answer due to insufficient understanding of the underlying information. Recent studies reveal that while LLMs can detect unanswerable questions, they struggle to assist users in reformulating these questions. Even advanced models like GPT-3.5 demonstrate limited effectiveness in this regard. To address this limitation, we propose DRS: Deep Question Reformulation with Structured Output, a novel zero-shot method aimed at enhancing LLMs ability to assist users in reformulating questions to extract relevant information from new documents. DRS combines the strengths of LLMs with a DFS-based algorithm to iteratively explore potential entity combinations and constrain outputs using predefined entities. This structured approach significantly enhances the reformulation capabilities of LLMs. Comprehensive experimental evaluations demonstrate that DRS improves the reformulation accuracy of GPT-3.5 from 23.03% to 70.42%, while also enhancing the performance of open-source models, such as Gemma2-9B, from 26.35% to 56.75%.

翻译：问答能力代表了大型语言模型（LLM）的核心能力。然而，当用户在文本中遇到不熟悉的知识时，由于对底层信息理解不足，他们提出的问题往往是文本本身无法回答的。近期研究表明，尽管LLM能够检测出不可回答的问题，但在协助用户重构这些问题方面却存在困难。即使是像GPT-3.5这样的先进模型，在此方面的有效性也相当有限。为了解决这一局限，我们提出了DRS：基于结构化输出的深度问题重构，这是一种新颖的零样本方法，旨在增强LLM协助用户重构问题以从新文档中提取相关信息的能力。DRS结合了LLM的优势与一种基于深度优先搜索（DFS）的算法，以迭代方式探索潜在的实体组合，并利用预定义的实体来约束输出。这种结构化方法显著提升了LLM的重构能力。全面的实验评估表明，DRS将GPT-3.5的重构准确率从23.03%提升至70.42%，同时也提升了开源模型（如Gemma2-9B）的性能，使其准确率从26.35%提高至56.75%。