We propose a new paradigm to help Large Language Models (LLMs) generate more accurate factual knowledge without retrieving from an external corpus, called RECITation-augmented gEneration (RECITE). Different from retrieval-augmented language models that retrieve relevant documents before generating the outputs, given an input, RECITE first recites one or several relevant passages from LLMs' own memory via sampling, and then produces the final answers. We show that RECITE is a powerful paradigm for knowledge-intensive NLP tasks. Specifically, we show that by utilizing recitation as the intermediate step, a recite-and-answer scheme can achieve new state-of-the-art performance in various closed-book question answering (CBQA) tasks. In experiments, we verify the effectiveness of \method~on four pre-trained models (PaLM, UL2, OPT, and Codex) and three CBQA tasks (Natural Questions, TriviaQA, and HotpotQA). Our code is available at "https://github.com/Edward-Sun/RECITE".

翻译：我们提出一种新范式，帮助大型语言模型（LLMs）无需从外部语料库检索即可生成更准确的事实知识，称为背诵增强生成（RECITE）。与检索增强型语言模型在生成输出前检索相关文档不同，给定输入后，RECITE首先通过采样从LLMs自身记忆中背诵一个或多个相关段落，随后生成最终答案。我们证明RECITE是处理知识密集型自然语言处理任务的有效范式。具体而言，通过将背诵作为中间步骤，背诵-回答方案可在多种闭卷问答（CBQA）任务中实现新的最先进性能。实验中，我们在四个预训练模型（PaLM、UL2、OPT和Codex）及三个CBQA任务（Natural Questions、TriviaQA和HotpotQA）上验证了\method~的有效性。我们的代码可在“https://github.com/Edward-Sun/RECITE”获取。