The advancement of Multi-modal Pre-training highlights the necessity for a robust Multi-Modal Knowledge Graph (MMKG) representation learning framework. This framework is crucial for integrating structured knowledge into multi-modal Large Language Models (LLMs) at scale, aiming to alleviate issues like knowledge misconceptions and multi-modal hallucinations. In this work, to evaluate models' ability to accurately embed entities within MMKGs, we focus on two widely researched tasks: Multi-modal Knowledge Graph Completion (MKGC) and Multi-modal Entity Alignment (MMEA). Building on this foundation, we propose a novel SNAG method that utilizes a Transformer-based architecture equipped with modality-level noise masking for the robust integration of multi-modal entity features in KGs. By incorporating specific training objectives for both MKGC and MMEA, our approach achieves SOTA performance across a total of ten datasets (three for MKGC and seven for MEMA), demonstrating its robustness and versatility. Besides, SNAG can not only function as a standalone model but also enhance other existing methods, providing stable performance improvements. Our code and data are available at: https://github.com/zjukg/SNAG.

翻译：多模态预训练的进展凸显了构建一个鲁棒的多模态知识图谱表示学习框架的必要性。该框架对于将结构化知识大规模集成到多模态大型语言模型中至关重要，旨在缓解知识误解和多模态幻觉等问题。在本研究中，为评估模型准确嵌入多模态知识图谱中实体的能力，我们聚焦于两个广泛研究的任务：多模态知识图谱补全与多模态实体对齐。基于此，我们提出了一种新颖的SNAG方法，该方法采用基于Transformer的架构，配备模态级噪声掩码机制，以鲁棒地整合知识图谱中的多模态实体特征。通过为多模态知识图谱补全和多模态实体对齐设计特定的训练目标，我们的方法在总计十个数据集（三个用于多模态知识图谱补全，七个用于多模态实体对齐）上实现了最先进的性能，证明了其鲁棒性和通用性。此外，SNAG不仅可作为独立模型运行，还能增强其他现有方法，提供稳定的性能提升。我们的代码和数据可在以下链接获取：https://github.com/zjukg/SNAG。