Graph-based Retrieval-Augmented Generation (RAG) has proven effective in integrating external knowledge into large language models (LLMs), improving their factual accuracy, adaptability, interpretability, and trustworthiness. A number of graph-based RAG methods have been proposed in the literature. However, these methods have not been systematically and comprehensively compared under the same experimental settings. In this paper, we first summarize a unified framework to incorporate all graph-based RAG methods from a high-level perspective. We then extensively compare representative graph-based RAG methods over a range of questing-answering (QA) datasets -- from specific questions to abstract questions -- and examine the effectiveness of all methods, providing a thorough analysis of graph-based RAG approaches. As a byproduct of our experimental analysis, we are also able to identify new variants of the graph-based RAG methods over specific QA and abstract QA tasks respectively, by combining existing techniques, which outperform the state-of-the-art methods. Finally, based on these findings, we offer promising research opportunities. We believe that a deeper understanding of the behavior of existing methods can provide new valuable insights for future research.
翻译:基于图的检索增强生成(Graph-based RAG)已被证明能有效将外部知识整合到大语言模型(LLMs)中,从而提升其事实准确性、适应性、可解释性和可信度。尽管文献中已提出多种基于图的RAG方法,但这些方法尚未在相同实验设置下进行系统全面的比较。本文首先从高层面总结了一个统一框架,以囊括所有基于图的RAG方法。随后,我们在一系列问答(QA)数据集上——从具体问题到抽象问题——广泛比较了代表性的基于图的RAG方法,考察了所有方法的有效性,并提供了对基于图的RAG方法的全面分析。作为实验分析的副产品,我们还通过结合现有技术,分别在特定问答和抽象问答任务中识别出基于图的RAG方法的新变体,这些变体优于当前最先进的方法。最后,基于这些发现,我们提出了有前景的研究方向。我们相信,对现有方法行为的更深理解能为未来研究提供新的宝贵见解。