A large number of studies have emerged for Multimodal Knowledge Graph Completion (MKGC) to predict the missing links in MKGs. However, fewer studies have been proposed to study the inductive MKGC (IMKGC) involving emerging entities unseen during training. Existing inductive approaches focus on learning textual entity representations, which neglect rich semantic information in visual modality. Moreover, they focus on aggregating structural neighbors from existing KGs, which of emerging entities are usually limited. However, the semantic neighbors are decoupled from the topology linkage and usually imply the true target entity. In this paper, we propose the IMKGC task and a semantic neighbor retrieval-enhanced IMKGC framework CMR, where the contrast brings the helpful semantic neighbors close, and then the memorize supports semantic neighbor retrieval to enhance inference. Specifically, we first propose a unified cross-modal contrastive learning to simultaneously capture the textual-visual and textual-textual correlations of query-entity pairs in a unified representation space. The contrastive learning increases the similarity of positive query-entity pairs, therefore making the representations of helpful semantic neighbors close. Then, we explicitly memorize the knowledge representations to support the semantic neighbor retrieval. At test time, we retrieve the nearest semantic neighbors and interpolate them to the query-entity similarity distribution to augment the final prediction. Extensive experiments validate the effectiveness of CMR on three inductive MKGC datasets. Codes are available at https://github.com/OreOZhao/CMR.
翻译:为预测多模态知识图谱(MKGs)中的缺失链接,大量关于多模态知识图谱补全(MKGC)的研究相继涌现。然而,针对涉及训练期间未见新兴实体的归纳式多模态知识图谱补全(IMKGC)的研究则相对较少。现有的归纳方法主要侧重于学习文本实体表示,忽略了视觉模态中丰富的语义信息。此外,这些方法聚焦于聚合来自现有知识图谱的结构邻居,而新兴实体的结构邻居通常有限。然而,语义邻居与拓扑链接解耦,且通常暗示着真实的目标实体。本文提出了IMKGC任务以及一个语义邻居检索增强的IMKGC框架CMR,其中对比学习使有帮助的语义邻居彼此接近,而后记忆机制支持语义邻居检索以增强推理。具体而言,我们首先提出一种统一的跨模态对比学习,以在统一的表示空间中同时捕获查询-实体对的文本-视觉与文本-文本相关性。该对比学习增加了正例查询-实体对的相似性,从而使有帮助的语义邻居的表示彼此接近。接着,我们显式地记忆知识表示以支持语义邻居检索。在测试阶段,我们检索最接近的语义邻居,并将其插值到查询-实体相似度分布中,以增强最终预测。大量实验在三个归纳式MKGC数据集上验证了CMR的有效性。代码可在 https://github.com/OreOZhao/CMR 获取。