In recent years, large language models (LLMs) have made remarkable achievements in various domains. However, the untimeliness and cost of knowledge updates coupled with hallucination issues of LLMs have curtailed their applications in knowledge intensive tasks, where retrieval augmented generation (RAG) can be of help. Nevertheless, existing retrieval augmented models typically use similarity as a bridge between queries and documents and follow a retrieve then read procedure. In this work, we argue that similarity is not always the panacea and totally relying on similarity would sometimes degrade the performance of retrieval augmented generation. To this end, we propose MetRag, a Multi layEred Thoughts enhanced Retrieval Augmented Generation framework. To begin with, beyond existing similarity oriented thought, we embrace a small scale utility model that draws supervision from an LLM for utility oriented thought and further come up with a smarter model by comprehensively combining the similarity and utility oriented thoughts. Furthermore, given the fact that the retrieved document set tends to be huge and using them in isolation makes it difficult to capture the commonalities and characteristics among them, we propose to make an LLM as a task adaptive summarizer to endow retrieval augmented generation with compactness-oriented thought. Finally, with multi layered thoughts from the precedent stages, an LLM is called for knowledge augmented generation. Extensive experiments on knowledge-intensive tasks have demonstrated the superiority of MetRag.

翻译：近年来，大语言模型（LLM）在各个领域取得了显著成就。然而，其知识更新的滞后性与高昂成本，加之LLM本身存在的幻觉问题，限制了它们在知识密集型任务中的应用，而检索增强生成（RAG）技术可为此提供助力。尽管如此，现有的检索增强模型通常以相似性作为查询与文档之间的桥梁，并遵循“先检索后阅读”的流程。本文认为，相似性并非总是万灵药，完全依赖相似性有时反而会损害检索增强生成的性能。为此，我们提出了MetRag，一个多层思维增强的检索增强生成框架。首先，在现有面向相似性的思维之外，我们引入了一个从小规模效用模型中汲取、并由LLM监督的面向效用的思维，进而通过综合结合面向相似性与效用的思维，提出了一个更智能的模型。此外，鉴于检索到的文档集往往规模庞大，且孤立使用它们难以捕捉其间的共性与特性，我们提出将LLM作为任务自适应的摘要生成器，从而赋予检索增强生成面向紧凑性的思维。最后，利用前述阶段产生的多层思维，调用LLM进行知识增强的生成。在知识密集型任务上进行的大量实验证明了MetRag的优越性。