Despite the significant progress of large language models (LLMs) in various tasks, they often produce factual errors due to their limited internal knowledge. Retrieval-Augmented Generation (RAG), which enhances LLMs with external knowledge sources, offers a promising solution. However, these methods can be misled by irrelevant paragraphs in retrieved documents. Due to the inherent uncertainty in LLM generation, inputting the entire document may introduce off-topic information, causing the model to deviate from the central topic and affecting the relevance of the generated content. To address these issues, we propose the Retrieve-Plan-Generation (RPG) framework. RPG generates plan tokens to guide subsequent generation in the plan stage. In the answer stage, the model selects relevant fine-grained paragraphs based on the plan and uses them for further answer generation. This plan-answer process is repeated iteratively until completion, enhancing generation relevance by focusing on specific topics. To implement this framework efficiently, we utilize a simple but effective multi-task prompt-tuning method, enabling the existing LLMs to handle both planning and answering. We comprehensively compare RPG with baselines across 5 knowledge-intensive generation tasks, demonstrating the effectiveness of our approach.

翻译：尽管大语言模型（LLM）在各种任务中取得了显著进展，但由于其内部知识有限，常常产生事实性错误。检索增强生成（RAG）通过外部知识源增强LLM能力，提供了一种有前景的解决方案。然而，这些方法可能被检索文档中的无关段落误导。由于LLM生成固有的不确定性，输入完整文档可能引入偏离主题的信息，导致模型偏离核心主题并影响生成内容的相关性。为解决这些问题，我们提出了检索-规划-生成（RPG）框架。RPG在规划阶段生成规划标记以指导后续生成。在应答阶段，模型根据规划选择相关的细粒度段落，并利用这些段落进行进一步的答案生成。这种规划-应答过程迭代重复直至完成，通过聚焦特定主题来增强生成相关性。为高效实现该框架，我们采用了一种简单而有效的多任务提示调优方法，使现有LLM能够同时处理规划与应答任务。我们在5项知识密集型生成任务上全面比较了RPG与基线方法，验证了所提方法的有效性。