Large Language Models (LLMs) have shown remarkable generalization capability with exceptional performance in various language modeling tasks. However, they still exhibit inherent limitations in precisely capturing and returning grounded knowledge. While existing work has explored utilizing knowledge graphs to enhance language modeling via joint training and customized model architectures, applying this to LLMs is problematic owing to their large number of parameters and high computational cost. In addition, how to leverage the pre-trained LLMs and avoid training a customized model from scratch remains an open question. In this work, we propose Graph Neural Prompting (GNP), a novel plug-and-play method to assist pre-trained LLMs in learning beneficial knowledge from KGs. GNP encompasses various designs, including a standard graph neural network encoder, a cross-modality pooling module, a domain projector, and a self-supervised link prediction objective. Extensive experiments on multiple datasets demonstrate the superiority of GNP on both commonsense and biomedical reasoning tasks across different LLM sizes and settings.

翻译：大语言模型（LLMs）在各类语言建模任务中展现出卓越的泛化能力和出色性能。然而，它们在精确捕获和返回接地知识方面仍存在固有局限性。现有研究探索了通过联合训练和定制化模型架构利用知识图谱增强语言建模，但由于LLMs参数规模庞大且计算成本高昂，将其直接应用于LLMs存在困难。此外，如何利用预训练LLMs并避免从零开始训练定制化模型仍是一个开放性问题。本研究提出图神经提示方法（GNP），一种新颖的即插即用方法，旨在协助预训练LLMs从知识图谱中学习有益知识。GNP包含多种设计，包括标准图神经网络编码器、跨模态池化模块、领域投影器以及自监督链路预测目标。在多个数据集上的大量实验表明，GNP在常识推理和生物医学推理任务中，在不同LLM规模及设置下均展现出显著优势。