Multimodal Knowledge Graphs (MKGs) extend traditional knowledge graphs by incorporating visual and textual modalities, enabling richer and more expressive entity representations. However, existing MKGs often suffer from incompleteness, which hinder their effectiveness in downstream tasks. Therefore, multimodal knowledge graph completion (MKGC) task is receiving increasing attention. While large language models (LLMs) have shown promise for knowledge graph completion (KGC), their application to the multimodal setting remains underexplored. Moreover, applying Multimodal Large Language Models (MLLMs) to the task of MKGC introduces significant challenges: (1) the large number of image tokens per entity leads to semantic noise and modality conflicts, and (2) the high computational cost of processing large token inputs. To address these issues, we propose Efficient Lightweight Multimodal Large Language Models (ELMM) for MKGC. ELMM proposes a Multi-view Visual Token Compressor (MVTC) based on multi-head attention mechanism, which adaptively compresses image tokens from both textual and visual views, thereby effectively reducing redundancy while retaining necessary information and avoiding modality conflicts. Additionally, we design an attention pruning strategy to remove redundant attention layers from MLLMs, thereby significantly reducing the inference cost. We further introduce a linear projection to compensate for the performance degradation caused by pruning. Extensive experiments on four benchmark datasets demonstrate that ELMM achieves state-of-the-art performance.

翻译：多模态知识图谱（MKGs）通过整合视觉与文本模态扩展了传统知识图谱，实现了更丰富、更具表现力的实体表征。然而，现有MKGs常存在不完整性问题，制约了其在下游任务中的有效性。因此，多模态知识图谱补全（MKGC）任务日益受到关注。尽管大语言模型（LLMs）在知识图谱补全（KGC）中展现出潜力，但其在多模态场景中的应用仍待深入探索。此外，将多模态大语言模型（MLLMs）应用于MKGC任务面临显著挑战：（1）每个实体对应的图像令牌数量庞大，易导致语义噪声与模态冲突；（2）处理大规模令牌输入的计算成本高昂。为应对这些问题，我们提出用于MKGC的高效轻量级多模态大语言模型（ELMM）。ELMM提出一种基于多头注意力机制的多视角视觉令牌压缩器（MVTC），该模块从文本与视觉双视角自适应压缩图像令牌，从而在保留必要信息、避免模态冲突的同时有效减少冗余。此外，我们设计了一种注意力剪枝策略，以移除MLLMs中的冗余注意力层，显著降低推理成本。我们进一步引入线性投影机制以补偿剪枝导致的性能损失。在四个基准数据集上的大量实验表明，ELMM实现了最先进的性能。