While large language models exhibit remarkable performance in the Question Answering task, they are susceptible to hallucinations. Challenges arise when these models grapple with understanding multi-hop relations in complex questions or lack the necessary knowledge for a comprehensive response. To address this issue, we introduce the "Decompose-and-Query" framework (D&Q). This framework guides the model to think and utilize external knowledge similar to ReAct, while also restricting its thinking to reliable information, effectively mitigating the risk of hallucinations. Experiments confirm the effectiveness of D&Q: On our ChitChatQA dataset, D&Q does not lose to ChatGPT in 67% of cases; on the HotPotQA question-only setting, D&Q achieved an F1 score of 59.6%. Our code is available at https://github.com/alkaidpku/DQ-ToolQA.

翻译：虽然大语言模型在问答任务中展现出卓越性能，但它们容易产生幻觉。当这些模型难以理解复杂问题中的多跳关系，或缺乏提供全面响应所需的知识时，挑战便会浮现。为解决这一问题，我们提出了"分解与查询"框架（D&Q）。该框架引导模型像ReAct一样进行思考并利用外部知识，同时将其思考限制在可靠信息内，有效降低幻觉风险。实验证实了D&Q的有效性：在我们的ChitChatQA数据集上，D&Q在67%的情况下不逊于ChatGPT；在HotPotQA的仅问题设定下，D&Q取得了59.6%的F1分数。我们的代码已开源在https://github.com/alkaidpku/DQ-ToolQA。