With the wide application of Large Language Models (LLMs) such as ChatGPT, how to make the contents generated by LLM accurate and credible becomes very important, especially in complex knowledge-intensive tasks. In this paper, we propose a novel framework called Search-in-the-Chain (SearChain) to improve the accuracy, credibility and traceability of LLM-generated content for multi-hop question answering, which is a typical complex knowledge-intensive task. SearChain is a framework that deeply integrates LLM and information retrieval (IR). In SearChain, LLM constructs a chain-of-query, which is the decomposition of the multi-hop question. Each node of the chain is a query-answer pair consisting of an IR-oriented query and the answer generated by LLM for this query. IR verifies, completes, and traces the information of each node of the chain, so as to guide LLM to construct the correct chain-of-query, and finally answer the multi-hop question. SearChain makes LLM change from trying to give a answer to trying to construct the chain-of-query when faced with the multi-hop question, which can stimulate the knowledge-reasoning ability and provides the interface for IR to be deeply involved in reasoning process of LLM. IR interacts with each node of chain-of-query of LLM. It verifies the information of the node and provides the unknown knowledge to LLM, which ensures the accuracy of the whole chain in the process of LLM generating the answer. Besides, the contents returned by LLM to the user include not only the final answer but also the reasoning process for the question, that is, the chain-of-query and the supporting documents retrieved by IR for each node of the chain, which improves the credibility and traceability of the contents generated by LLM. Experimental results show SearChain outperforms related baselines on four multi-hop question-answering datasets.

翻译：随着ChatGPT等大型语言模型（LLMs）的广泛应用，如何使LLM生成的内容准确且可信变得至关重要，尤其是在复杂的知识密集型任务中。本文提出一种名为Search-in-the-Chain（SearChain）的新型框架，旨在提升针对多跳问答这一典型复杂知识密集型任务的LLM生成内容的准确性、可信度与可追溯性。SearChain是一种深度融合LLM与信息检索（IR）的框架。在该框架中，LLM构建一条查询链（chain-of-query），即对多跳问题的分解。链中的每个节点是一个由面向IR的查询及LLM针对该查询生成的答案组成的查询-答案对。IR对链中各节点的信息进行验证、补全与追溯，从而引导LLM构建正确的查询链，并最终回答多跳问题。SearChain使LLM在面对多跳问题时，从试图直接给出答案转变为尝试构建查询链，这既能激发其知识推理能力，又为IR深度参与LLM推理过程提供了接口。IR与LLM查询链的每个节点进行交互，验证节点信息并向LLM提供未知知识，从而确保LLM生成答案过程中整个链条的准确性。此外，LLM返回给用户的内容不仅包括最终答案，还包含针对问题的推理过程，即查询链以及IR为链中每个节点检索到的支持文档，这提升了LLM生成内容的可信度与可追溯性。实验结果表明，SearChain在四个多跳问答数据集上均优于相关基线方法。