Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating additional information from retrieval. However, studies have shown that LLMs still face challenges in effectively using the retrieved information, even ignoring it or being misled by it. The key reason is that the training of LLMs does not clearly make LLMs learn how to utilize input retrieved texts with varied quality. In this paper, we propose a novel perspective that considers the role of LLMs in RAG as ``Information Refiner'', which means that regardless of correctness, completeness, or usefulness of retrieved texts, LLMs can consistently integrate knowledge within the retrieved texts and model parameters to generate the texts that are more concise, accurate, and complete than the retrieved texts. To this end, we propose an information refinement training method named InFO-RAG that optimizes LLMs for RAG in an unsupervised manner. InFO-RAG is low-cost and general across various tasks. Extensive experiments on zero-shot prediction of 11 datasets in diverse tasks including Question Answering, Slot-Filling, Language Modeling, Dialogue, and Code Generation show that InFO-RAG improves the performance of LLaMA2 by an average of 9.39\% relative points. InFO-RAG also shows advantages in in-context learning and robustness of RAG.

翻译：检索增强生成（RAG）通过引入检索到的额外信息来增强大型语言模型（LLMs）的能力。然而，研究表明，LLMs在有效利用检索信息方面仍面临挑战，甚至可能忽略或受其误导。其根本原因在于，LLMs的训练过程并未明确使其学会如何处理质量参差的输入检索文本。本文提出一种新颖视角，将LLMs在RAG中的角色定义为"信息精炼器"（Information Refiner），即无论检索文本的正确性、完整性或有用性如何，LLMs都能持续整合检索文本与模型参数中的知识，生成比检索文本更简洁、准确、完整的文本。为此，我们提出一种名为InFO-RAG的信息精炼训练方法，以无监督方式优化LLMs在RAG中的性能。InFO-RAG成本低廉且可泛化至多种任务。在涵盖问答、槽填充、语言建模、对话及代码生成等11个数据集的零样本预测实验中，InFO-RAG使LLaMA2的平均性能相对提升9.39%。此外，InFO-RAG在上下文学习能力及RAG鲁棒性方面也展现出优势。