With the wide application of Large Language Models (LLMs) such as ChatGPT, how to make the contents generated by LLM accurate and credible becomes very important, especially in complex knowledge-intensive tasks. In this paper, we propose a novel framework called Search-in-the-Chain (SearChain) to improve the accuracy, credibility and traceability of LLM-generated content for multi-hop question answering, which is a typical complex knowledge-intensive task. SearChain is a framework that deeply integrates LLM and information retrieval (IR). In SearChain, LLM constructs a chain-of-query, which is the decomposition of the multi-hop question. Each node of the chain is a query-answer pair consisting of an IR-oriented query and the answer generated by LLM for this query. IR verifies, completes, and traces the information of each node of the chain, so as to guide LLM to construct the correct chain-of-query, and finally answer the multi-hop question. SearChain makes LLM change from trying to give a answer to trying to construct the chain-of-query when faced with the multi-hop question, which can stimulate the knowledge-reasoning ability and provides the interface for IR to be deeply involved in reasoning process of LLM. IR interacts with each node of chain-of-query of LLM. It verifies the information of the node and provides the unknown knowledge to LLM, which ensures the accuracy of the whole chain in the process of LLM generating the answer. Besides, the contents returned by LLM to the user include not only the final answer but also the reasoning process for the question, that is, the chain-of-query and the supporting documents retrieved by IR for each node of the chain, which improves the credibility and traceability of the contents generated by LLM. Experimental results show SearChain outperforms related baselines on four multi-hop question-answering datasets.

翻译：随着ChatGPT等大型语言模型(LLM)的广泛应用，如何使LLM生成的内容准确可信变得至关重要，尤其在复杂知识密集型任务中。本文提出一种名为"搜索链"(Search-in-the-Chain, SearChain)的新型框架，旨在提升LLM生成内容在多跳问答（典型复杂知识密集型任务）中的准确性、可信度与可溯源性。SearChain是深度融合LLM与信息检索(IR)的框架。在该框架中，LLM构建查询链（即多跳问题的分解），链中各节点为查询-答案对，包含面向IR的查询及LLM对该查询生成的答案。IR对链中各节点的信息进行验证、补全与溯源，从而引导LLM构建正确的查询链，最终解答多跳问题。SearChain使LLM在面对多跳问题时，从试图直接给出答案转变为构建查询链，这既能激发知识推理能力，也为IR深度参与LLM推理过程提供了接口。IR与LLM查询链的每个节点交互，验证节点信息并向LLM提供未知知识，确保LLM生成答案过程中整条链的准确性。此外，LLM返回给用户的内容不仅包含最终答案，更包括问题的推理过程，即查询链及IR为链中各节点检索的支持文档，从而提升LLM生成内容的可信度与可溯源性。实验结果表明，SearChain在四个多跳问答数据集上均显著优于相关基线方法。