Multi-modal knowledge graph completion (MMKGC) aims to predict the missing triples in the multi-modal knowledge graphs by incorporating structural, visual, and textual information of entities into the discriminant models. The information from different modalities will work together to measure the triple plausibility. Existing MMKGC methods overlook the imbalance problem of modality information among entities, resulting in inadequate modal fusion and inefficient utilization of the raw modality information. To address the mentioned problems, we propose Adaptive Multi-modal Fusion and Modality Adversarial Training (AdaMF-MAT) to unleash the power of imbalanced modality information for MMKGC. AdaMF-MAT achieves multi-modal fusion with adaptive modality weights and further generates adversarial samples by modality-adversarial training to enhance the imbalanced modality information. Our approach is a co-design of the MMKGC model and training strategy which can outperform 19 recent MMKGC methods and achieve new state-of-the-art results on three public MMKGC benchmarks. Our code and data have been released at https://github.com/zjukg/AdaMF-MAT.

翻译：多模态知识图谱补全(MMKGC)旨在通过将实体的结构、视觉和文本信息融入判别模型，预测多模态知识图谱中缺失的三元组。不同模态的信息将协同工作以衡量三元组的合理性。现有MMKGC方法忽视了实体间模态信息的不平衡问题，导致模态融合不充分以及原始模态信息利用效率低下。为解决上述问题，我们提出自适应多模态融合与模态对抗训练(AdaMF-MAT)，以释放不平衡模态信息在MMKGC中的潜力。AdaMF-MAT通过自适应模态权重实现多模态融合，并进一步通过模态对抗训练生成对抗样本来增强不平衡的模态信息。我们的方法是一种MMKGC模型与训练策略的协同设计，能够超越19种近期MMKGC方法，在三个公开MMKGC基准上取得新的最优结果。我们的代码和数据已发布于https://github.com/zjukg/AdaMF-MAT。