Knowledge graphs (KGs) are the cornerstone of the semantic web, offering up-to-date representations of real-world entities and relations. Yet large language models (LLMs) remain largely static after pre-training, causing their internal knowledge to become outdated and limiting their utility in time-sensitive web applications. To bridge this gap between dynamic knowledge and static models, a prevalent approach is to enhance LLMs with KGs. However, prevailing methods typically rely on parameter-invasive fine-tuning, which risks catastrophic forgetting and often degrades LLMs' general capabilities. Moreover, their static integration frameworks cannot keep pace with the continuous evolution of real-world KGs, hindering their deployment in dynamic web environments. To bridge this gap, we introduce KGA (\textit{\underline{K}nowledge \underline{G}raph-guided \underline{A}ttention}), a novel framework that dynamically integrates external KGs into LLMs exclusively at inference-time without any parameter modification. Inspired by research on neuroscience, we rewire the self-attention module by innovatively introducing two synergistic pathways: a \textit{bottom-up knowledge fusion} pathway and a \textit{top-down attention guidance} pathway. The \textit{bottom-up pathway} dynamically integrates external knowledge into input representations via input-driven KG fusion, which is akin to the \textit{stimulus-driven attention process} in the human brain. Complementarily, the \textit{top-down pathway} aims to assess the contextual relevance of each triple through a \textit{goal-directed verification process}, thereby suppressing task-irrelevant signals and amplifying knowledge-relevant patterns. By synergistically combining these two pathways, our method supports real-time knowledge fusion. Extensive experiments on four benchmarks verify KGA's strong fusion performance and efficiency.

翻译：知识图谱（KGs）是语义网的基石，提供了现实世界实体与关系的最新表示。然而，大型语言模型（LLMs）在预训练后基本保持静态，导致其内部知识逐渐过时，限制了其在时效性强的网络应用中的效用。为弥合动态知识与静态模型之间的鸿沟，一种主流方法是用知识图谱增强LLMs。然而，现有方法通常依赖于参数侵入式的微调，这可能导致灾难性遗忘，并常常削弱LLMs的通用能力。此外，其静态集成框架无法跟上现实世界知识图谱的持续演化，阻碍了其在动态网络环境中的部署。为填补这一空白，我们提出了KGA（知识图谱引导注意力），一种新颖的框架，仅在推理时动态地将外部知识图谱集成到LLMs中，无需任何参数修改。受神经科学研究的启发，我们通过创新性地引入两条协同通路来重构自注意力模块：一条自下而上的知识融合通路和一条自上而下的注意力引导通路。自下而上通路通过输入驱动的知识图谱融合，动态地将外部知识整合到输入表示中，这类似于人脑中的刺激驱动注意过程。互补地，自上而下通路旨在通过目标导向的验证过程评估每个三元组的上下文相关性，从而抑制任务无关信号并增强知识相关模式。通过协同结合这两条通路，我们的方法支持实时知识融合。在四个基准测试上的大量实验验证了KGA强大的融合性能与效率。