Question answering is a fundamental capability of large language models (LLMs). However, when people encounter completely new knowledge texts, they often ask questions that the text cannot answer due to a lack of understanding of the knowledge. Recent research shows that large language models identify the unanswerability of questions, but they lack the ability to help people reformulate their questions. Even powerful models like GPT-3.5 perform poorly in this regard. To enhance the ability of LLMs to assist humans in reformulating questions to extract relevant knowledge from new documents, we propose a zero-shot method called DRS: Deep Question Reformulation With Structured Output. Our proposed method leverages large language models and the DFS-based algorithm to iteratively search for possible entity combinations and constrain the output with certain entities, effectively improving the capabilities of large language models in this area. Extensive experimental results show that our zero-shot DRS method significantly improves the reformulation accuracy of GPT-3.5 from 23.03% to 70.42% and effectively improves the score of open-source large language models, such as Gemma2-9B, from 26.35% to 56.75%.

翻译：问答是大语言模型（LLM）的一项基本能力。然而，当人们遇到全新的知识文本时，由于对知识缺乏了解，他们提出的问题往往是文本无法回答的。近期研究表明，大语言模型能够识别问题的不可回答性，但它们缺乏帮助人们重述问题的能力。即使是像GPT-3.5这样强大的模型，在这方面也表现不佳。为了增强大语言模型协助人类重述问题、从新文档中提取相关知识的能力，我们提出了一种称为DRS的零样本方法：基于结构化输出的深度问题重述。我们提出的方法利用大语言模型和基于DFS的算法，迭代搜索可能的实体组合，并通过特定实体约束输出，有效提升了大语言模型在该领域的能力。大量实验结果表明，我们的零样本DRS方法将GPT-3.5的重述准确率从23.03%显著提升至70.42%，并有效提升了开源大语言模型（如Gemma2-9B）的得分，从26.35%提高至56.75%。