Retrieval-augmented generation (RAG) has proven effective in integrating knowledge into large language models (LLMs). However, conventional RAGs struggle to capture complex relationships between pieces of knowledge, limiting their performance in intricate reasoning that requires integrating knowledge from multiple sources. Recently, graph-enhanced retrieval augmented generation (GraphRAG) builds graph structure to explicitly model these relationships, enabling more effective and efficient retrievers. Nevertheless, its performance is still hindered by the noise and incompleteness within the graph structure. To address this, we introduce GFM-RAG, a novel graph foundation model (GFM) for retrieval augmented generation. GFM-RAG is powered by an innovative graph neural network that reasons over graph structure to capture complex query-knowledge relationships. The GFM with 8M parameters undergoes a two-stage training process on large-scale datasets, comprising 60 knowledge graphs with over 14M triples and 700k documents. This results in impressive performance and generalizability for GFM-RAG, making it the first graph foundation model applicable to unseen datasets for retrieval without any fine-tuning required. Extensive experiments on three multi-hop QA datasets and seven domain-specific RAG datasets demonstrate that GFM-RAG achieves state-of-the-art performance while maintaining efficiency and alignment with neural scaling laws, highlighting its potential for further improvement.

翻译：检索增强生成（RAG）已被证明在将知识整合到大型语言模型（LLM）中是有效的。然而，传统的RAG方法难以捕捉知识片段之间的复杂关系，限制了其在需要整合多源知识的复杂推理任务中的性能。最近，图增强的检索增强生成（GraphRAG）通过构建图结构来显式建模这些关系，从而实现了更有效和高效的检索器。尽管如此，其性能仍受到图结构中噪声和不完整性的制约。为解决此问题，我们提出了GFM-RAG，一种用于检索增强生成的新型图基础模型（GFM）。GFM-RAG由一个创新的图神经网络驱动，该网络在图结构上进行推理以捕捉复杂的查询-知识关系。这个拥有800万参数的GFM在大规模数据集上经历了两阶段训练过程，该数据集包含60个知识图谱（超过1400万个三元组）和70万份文档。这使得GFM-RAG获得了令人印象深刻的性能和泛化能力，使其成为首个无需任何微调即可应用于未见数据集进行检索的图基础模型。在三个多跳问答数据集和七个特定领域RAG数据集上进行的大量实验表明，GFM-RAG实现了最先进的性能，同时保持了效率并符合神经缩放定律，突显了其进一步改进的潜力。