Large Language Models (LLMs) have made significant strides in information acquisition. However, their overreliance on potentially flawed parametric knowledge leads to hallucinations and inaccuracies, particularly when handling long-tail, domain-specific queries. Retrieval Augmented Generation (RAG) addresses this limitation by incorporating external, non-parametric knowledge. Nevertheless, the retrieved long-context documents often contain noisy, irrelevant information alongside vital knowledge, negatively diluting LLMs' attention. Inspired by the supportive role of essential concepts in individuals' reading comprehension, we propose a novel concept-based RAG framework with the Abstract Meaning Representation (AMR)-based concept distillation algorithm. The proposed algorithm compresses the cluttered raw retrieved documents into a compact set of crucial concepts distilled from the informative nodes of AMR by referring to reliable linguistic features. The concepts explicitly constrain LLMs to focus solely on vital information in the inference process. We conduct extensive experiments on open-domain question-answering datasets to empirically evaluate the proposed method's effectiveness. The results indicate that the concept-based RAG framework outperforms other baseline methods, particularly as the number of supporting documents increases, while also exhibiting robustness across various backbone LLMs. This emphasizes the distilled concepts are informative for augmenting the RAG process by filtering out interference information. To the best of our knowledge, this is the first work introducing AMR to enhance the RAG, presenting a potential solution to augment inference performance with semantic-based context compression.

翻译：大型语言模型（LLMs）在信息获取方面取得了显著进展。然而，它们过度依赖可能存在缺陷的参数化知识，导致在处理长尾、特定领域的查询时出现幻觉和不准确性。检索增强生成（RAG）通过引入外部非参数化知识解决了这一限制。然而，检索到的长上下文文档通常包含与关键知识混杂的噪声和无关信息，从而负面稀释了LLMs的注意力。受核心概念在个体阅读理解中支持作用的启发，我们提出了一种基于概念的新型RAG框架，其中包含基于抽象语义表示（AMR）的概念蒸馏算法。该算法通过参考可靠的 linguistic 特征，将杂乱无章的原始检索文档压缩为从AMR信息节点蒸馏出的紧凑关键概念集。这些概念显式地约束LLMs在推理过程中仅聚焦于重要信息。我们在开放域问答数据集上进行了广泛的实验，以实证评估所提方法的有效性。结果表明，基于概念的RAG框架优于其他基线方法，尤其是在支持文档数量增加时，同时在不同骨干LLMs上表现出鲁棒性。这强调了蒸馏出的概念通过过滤干扰信息来增强RAG过程的有效性。据我们所知，这是首个引入AMR来增强RAG的工作，为基于语义的上下文压缩增强推理性能提供了潜在解决方案。