With the wide application of Large Language Models (LLMs) such as ChatGPT, how to make the contents generated by LLM accurate and credible becomes very important, especially in complex knowledge-intensive tasks. In this paper, we propose a novel framework called Search-in-the-Chain (SearChain) to improve the accuracy, credibility and traceability of LLM-generated content for multi-hop question answering, which is a typical complex knowledge-intensive task. SearChain is a framework that deeply integrates LLM and information retrieval (IR). In SearChain, LLM constructs a chain-of-query, which is the decomposition of the multi-hop question. Each node of the chain is a query-answer pair consisting of an IR-oriented query and the answer generated by LLM for this query. IR verifies, completes, and traces the information of each node of the chain, so as to guide LLM to construct the correct chain-of-query, and finally answer the multi-hop question. SearChain makes LLM change from trying to give a answer to trying to construct the chain-of-query when faced with the multi-hop question, which can stimulate the knowledge-reasoning ability and provides the interface for IR to be deeply involved in reasoning process of LLM. IR interacts with each node of chain-of-query of LLM. It verifies the information of the node and provides the unknown knowledge to LLM, which ensures the accuracy of the whole chain in the process of LLM generating the answer. Besides, the contents returned by LLM to the user include not only the final answer but also the reasoning process for the question, that is, the chain-of-query and the supporting documents retrieved by IR for each node of the chain, which improves the credibility and traceability of the contents generated by LLM. Experimental results show SearChain outperforms related baselines on four multi-hop question-answering datasets.

翻译：随着ChatGPT等大语言模型的广泛应用，如何使LLM生成的内容准确且可信变得尤为重要，尤其是在复杂的知识密集型任务中。本文提出一种名为搜索链（SearChain）的新型框架，旨在提升多跳问答（一种典型的复杂知识密集型任务）中LLM生成内容的准确性、可信度和可追溯性。SearChain是一个深度融合大语言模型与信息检索（IR）的框架。在该框架中，LLM构建一个查询链，该链是对多跳问题的分解。链的每个节点由面向信息检索的查询及其对应的LLM生成答案组成的查询-答案对构成。信息检索对链中每个节点的信息进行验证、补充和追溯，从而引导LLM构建正确的查询链，最终回答多跳问题。SearChain使LLM在面对多跳问题时，从试图直接给出答案转变为尝试构建查询链，这能激发其知识推理能力，并为信息检索深度参与LLM推理过程提供接口。信息检索与LLM查询链的每个节点进行交互：它验证节点信息并向LLM提供未知知识，从而确保LLM生成答案过程中整个链的准确性。此外，LLM返回给用户的内容不仅包含最终答案，还包含问题的推理过程，即查询链以及IR为每个链节点检索到的支持文档，这显著提升了LLM生成内容的可信度和可追溯性。实验结果表明，SearChain在四个多跳问答数据集上均优于相关基线方法。