Current large language models (LLMs) can exhibit near-human levels of performance on many natural language-based tasks, including open-domain question answering. Unfortunately, at this time, they also convincingly hallucinate incorrect answers, so that responses to questions must be verified against external sources before they can be accepted at face value. In this paper, we report two simple experiments to automatically validate generated answers against a corpus. We base our experiments on questions and passages from the MS MARCO (V1) test collection, and a retrieval pipeline consisting of sparse retrieval, dense retrieval and neural rerankers. In the first experiment, we validate the generated answer in its entirety. After presenting a question to an LLM and receiving a generated answer, we query the corpus with the combination of the question + generated answer. We then present the LLM with the combination of the question + generated answer + retrieved answer, prompting it to indicate if the generated answer can be supported by the retrieved answer. In the second experiment, we consider the generated answer at a more granular level, prompting the LLM to extract a list of factual statements from the answer and verifying each statement separately. We query the corpus with each factual statement and then present the LLM with the statement and the corresponding retrieved evidence. The LLM is prompted to indicate if the statement can be supported and make necessary edits using the retrieved material. With an accuracy of over 80%, we find that an LLM is capable of verifying its generated answer when a corpus of supporting material is provided. However, manual assessment of a random sample of questions reveals that incorrect generated answers are missed by this verification process. While this verification process can reduce hallucinations, it can not entirely eliminate them.

翻译：当前大型语言模型（LLMs）在众多自然语言任务（包括开放域问答）上表现出近乎人类水平的性能。然而，目前它们也会令人信服地产生错误幻觉，因此必须借助外部来源验证回答，才能将其视为真实可信。本文报告了两项简单实验，旨在自动验证生成答案与语料库的一致性。实验基于MS MARCO（V1）测试集中的问题与段落，以及由稀疏检索、稠密检索和神经重排序器组成的检索流水线。在第一项实验中，我们对生成的完整答案进行验证：向LLM提出问题并获取生成答案后，以“问题+生成答案”的组合查询语料库；随后将“问题+生成答案+检索答案”的组合输入LLM，提示其判断生成答案是否可由检索答案支持。在第二项实验中，我们以更细粒度的方式处理生成答案：提示LLM从答案中提取事实陈述列表，并逐一验证每个陈述。针对每个事实陈述查询语料库，随后将陈述及其对应检索证据输入LLM，提示其判断陈述是否可被支持，并利用检索材料进行必要修正。实验发现，在提供支持性语料库的条件下，LLM能够以超过80%的准确率验证自身生成的答案。然而，对随机样本的人工评估显示，该验证过程可能遗漏错误的生成答案。尽管此验证方法能减少幻觉，但无法完全消除。