Multimodal Attributed Graphs (MAGs) have been widely adopted for modeling complex systems by integrating multi-modal information, such as text and images, on nodes. However, we identify a discrepancy between the implicit semantic structure induced by different modality embeddings and the explicit graph structure. For instance, neighbors in the explicit graph structure may be close in one modality but distant in another. Since existing methods typically perform message passing over the fixed explicit graph structure, they inadvertently aggregate dissimilar features, introducing modality-specific noise and impeding effective node representation learning. To address this, we propose OptiMAG, an Unbalanced Optimal Transport-based regularization framework. OptiMAG employs the Fused Gromov-Wasserstein distance to explicitly guide cross-modal structural consistency within local neighborhoods, effectively mitigating structural-semantic conflicts. Moreover, a KL divergence penalty enables adaptive handling of cross-modal inconsistencies. This framework can be seamlessly integrated into existing multimodal graph models, acting as an effective drop-in regularizer. Experiments demonstrate that OptiMAG consistently outperforms baselines across multiple tasks, ranging from graph-centric tasks (e.g., node classification, link prediction) to multimodal-centric generation tasks (e.g., graph2text, graph2image). The source code will be available upon acceptance.

翻译：多模态属性图（MAGs）通过整合节点上的多模态信息（如文本和图像），已被广泛用于对复杂系统进行建模。然而，我们发现由不同模态嵌入所诱导的隐式语义结构与显式图结构之间存在差异。例如，显式图结构中的邻居可能在某一模态中相近，但在另一模态中却相距甚远。由于现有方法通常在固定的显式图结构上进行消息传递，它们会无意中聚合不相似的特征，引入模态特定的噪声，从而阻碍有效的节点表示学习。为解决此问题，我们提出了OptiMAG，一个基于非平衡最优传输的正则化框架。OptiMAG采用融合Gromov-Wasserstein距离来显式引导局部邻域内的跨模态结构一致性，有效缓解结构-语义冲突。此外，一个KL散度惩罚项能够自适应地处理跨模态不一致性。该框架可以无缝集成到现有的多模态图模型中，作为一种有效的即插即用正则化器。实验表明，OptiMAG在从以图为中心的任务（例如节点分类、链接预测）到以多模态为中心的生成任务（例如图到文本、图到图像）等多种任务中，均持续优于基线方法。源代码将在论文被接受后公开。