Knowledge graph (KG) link prediction aims to infer new facts based on existing facts in the KG. Recent studies have shown that using the graph neighborhood of a node via graph neural networks (GNNs) provides more useful information compared to just using the query information. Conventional GNNs for KG link prediction follow the standard message-passing paradigm on the entire KG, which leads to superfluous computation, over-smoothing of node representations, and also limits their expressive power. On a large scale, it becomes computationally expensive to aggregate useful information from the entire KG for inference. To address the limitations of existing KG link prediction frameworks, we propose a novel retrieve-and-read framework, which first retrieves a relevant subgraph context for the query and then jointly reasons over the context and the query with a high-capacity reader. As part of our exemplar instantiation for the new framework, we propose a novel Transformer-based GNN as the reader, which incorporates graph-based attention structure and cross-attention between query and context for deep fusion. This simple yet effective design enables the model to focus on salient context information relevant to the query. Empirical results on two standard KG link prediction datasets demonstrate the competitive performance of the proposed method. Furthermore, our analysis yields valuable insights for designing improved retrievers within the framework.

翻译：知识图谱（KG）链接预测旨在基于KG中已有事实推断新事实。已有研究表明，相较于仅使用查询信息，通过图神经网络（GNNs）利用节点的图邻域信息能提供更有用的特征。传统用于知识图谱链接预测的GNN遵循在整个KG上的标准消息传递范式，这会导致冗余计算、节点表征过度平滑，并限制其表达能力。在大规模场景下，从整个KG中聚合有用信息进行推理在计算上变得昂贵。为解决现有知识图谱链接预测框架的局限性，我们提出一种新颖的检索-阅读框架：首先检索与查询相关的子图上下文，随后通过高容量阅读器对上下文和查询进行联合推理。作为该框架的实例化方案，我们提出一种基于Transformer的新型GNN作为阅读器，其融合了基于图的注意力结构以及查询与上下文间的交叉注意力以实现深度融合。这一简洁而有效的设计使模型能够聚焦于与查询相关的显著上下文信息。在两个标准知识图谱链接预测数据集上的实验结果验证了所提方法的竞争性表现。此外，我们的分析为在该框架内设计更优的检索器提供了宝贵见解。