Recent works in open-domain question answering (QA) have explored generating context passages from large language models (LLMs), replacing the traditional retrieval step in the QA pipeline. However, it is not well understood why generated passages can be more effective than retrieved ones. This study revisits the conventional formulation of QA and introduces the concept of knowledge corpus error. This error arises when the knowledge corpus used for retrieval is only a subset of the entire string space, potentially excluding more helpful passages that exist outside the corpus. LLMs may mitigate this shortcoming by generating passages in a larger space. We come up with an experiment of paraphrasing human-annotated gold context using LLMs to observe knowledge corpus error empirically. Our results across three QA benchmarks reveal an increased performance (10% - 13%) when using paraphrased passage, indicating a signal for the existence of knowledge corpus error. Our code is available at https://github.com/xfactlab/emnlp2023-knowledge-corpus-error

翻译：在开放域问答（QA）领域的最新研究中，利用大型语言模型（LLM）生成的上下文段落替代了传统问答流程中的检索步骤。然而，为何生成的段落可能比检索到的段落更有效，目前尚未得到充分理解。本研究重新审视了问答的传统范式，并引入了“知识语料库误差”这一概念。该误差源于用于检索的知识语料库仅是整个字符串空间的一个子集，可能遗漏了语料库外更具助益的段落。LLM通过生成更大空间内的段落，或可缓解这一缺陷。我们设计了一项实验，利用LLM对人工标注的黄金上下文进行释义，以实证观察知识语料库误差。在三个问答基准测试中的结果表明，使用释义后的段落使性能提升了10%-13%，这证明了知识语料库误差的存在。我们的代码已开源：https://github.com/xfactlab/emnlp2023-knowledge-corpus-error