Multi-modal knowledge graph completion (MMKGC) aims to automatically discover new knowledge triples in the given multi-modal knowledge graphs (MMKGs), which is achieved by collaborative modeling the structural information concealed in massive triples and the multi-modal features of the entities. Existing methods tend to focus on crafting elegant entity-wise multi-modal fusion strategies, yet they overlook the utilization of multi-perspective features concealed within the modalities under diverse relational contexts. To address this issue, we introduce a novel MMKGC framework with Mixture of Modality Knowledge experts (MoMoK for short) to learn adaptive multi-modal embedding under intricate relational contexts. We design relation-guided modality knowledge experts to acquire relation-aware modality embeddings and integrate the predictions from multi-modalities to achieve comprehensive decisions. Additionally, we disentangle the experts by minimizing their mutual information. Experiments on four public MMKG benchmarks demonstrate the outstanding performance of MoMoK under complex scenarios.
翻译:多模态知识图谱补全(MMKGC)旨在通过协同建模海量三元组中蕴含的结构信息与实体的多模态特征,自动发现给定多模态知识图谱(MMKG)中的新知识三元组。现有方法往往侧重于设计精巧的实体级多模态融合策略,却忽视了在不同关系语境下挖掘模态内部隐藏的多视角特征。为解决这一问题,我们提出一种新颖的混合模态知识专家(简称MoMoK)MMKGC框架,以学习复杂关系语境下的自适应多模态嵌入。我们设计了关系引导的模态知识专家来获取关系感知的模态嵌入,并通过整合多模态的预测结果实现综合决策。此外,我们通过最小化专家间的互信息来实现专家解耦。在四个公开MMKG基准数据集上的实验表明,MoMoK在复杂场景下具有卓越的性能。