Multimodal Emotion Recognition (MER) has attracted growing attention with the rapid advancement of human-computer interaction. However, different modalities exhibit substantial discrepancies in semantics, quality, and availability, leading to highly heterogeneous modality combinations and posing significant challenges to achieving consistent and reliable emotion understanding. To address this challenge, we propose the Modality-Aware Contrastive and Uncertainty-Regularized (MCUR) framework, which approaches MER from the perspective of representation consistency, aiming to enable robust emotion prediction across heterogeneous modality combinations. MCUR incorporates two core components: (1) Modality Combination-Based and Category-Based Contrastive Learning mechanism (MCB-CL), which encourages samples with the same emotion category and the same available modalities to be close in the representation space; and (2) Sample-wise Uncertainty-Guided Regularization (SUGR), which adaptively assigns sample-wise uncertain weights to samples to optimize training. Extensive experiments demonstrate that MCUR consistently outperforms existing methods, achieving average F1 gains of 2.2% on MOSI, 2.67% on MOSEI, and 4.37% on IEMOCAP.

翻译：多模态情感识别（Multimodal Emotion Recognition, MER）随着人机交互的快速发展而日益受到关注。然而，不同模态在语义、质量和可用性方面存在显著差异，导致高度异构的模态组合，对实现一致可靠的情感理解构成重大挑战。为解决这一难题，我们提出了模态感知对比与不确定性正则化框架（Modality-Aware Contrastive and Uncertainty-Regularized, MCUR），该框架从表示一致性的角度处理MER问题，旨在实现跨异构模态组合的鲁棒情感预测。MCUR包含两个核心组件：（1）基于模态组合与基于类别的对比学习机制（MCB-CL），该机制促使具有相同情感类别和相同可用模态的样本在表示空间中彼此靠近；（2）样本级不确定性引导正则化（SUGR），该机制自适应地为样本分配样本级不确定性权重以优化训练。大量实验表明，MCUR始终优于现有方法，在MOSI、MOSEI和IEMOCAP数据集上分别实现了平均F1值2.2%、2.67%和4.37%的提升。