Multimodal Intent Recognition (MIR) aims to understand complex user intentions by leveraging text, video, and audio signals. However, existing approaches face two key challenges: (1) overlooking intricate cross-modal interactions for distinguishing consistent and inconsistent cues, and (2) ineffectively modeling multimodal conflicts, leading to semantic cancellation. To address these, we propose a novel Cognitive Dual-Pathway Reasoning (CDPR) framework, which constructs a stable semantic foundation via the intuition pathway and mitigates high-level semantic conflicts through the reasoning pathway, cooperatively establishing deep semantic relations. Specifically, we first employ a representation disentanglement strategy to extract modality-invariant and specific features. Subsequently, the intuition pathway aggregates cross-modal consensus using shared features for solid global representations. The reasoning pathway introduces an inconsistency perception mechanism, combining semantic prototype matching with statistical probability calibration to precisely quantify conflict severity, and dynamically adjusting the weights between both pathways. Furthermore, a multi-view loss function is adopted to alleviate modality laziness and learn structured features at different stages. Extensive experiments on two benchmarks show that CDPR achieves SOTA performance and superior robustness in mitigating multimodal inconsistency. The code is available at https://github.com/Hebust-NLP/CDPR.
翻译:多模态意图识别(MIR)旨在利用文本、视频和音频信号理解复杂的用户意图。然而,现有方法面临两个关键挑战:(1)忽略区分一致与不一致线索所需的复杂跨模态交互,(2)对多模态冲突的建模效率低下,导致语义抵消。为解决这些问题,我们提出了一种新颖的认知双路径推理(CDPR)框架,该框架通过直觉路径构建稳定的语义基础,并通过推理路径缓解高层语义冲突,协作建立深层语义关联。具体而言,我们首先采用表示解耦策略提取模态不变特征与模态特定特征。随后,直觉路径利用共享特征聚合跨模态共识,形成稳固的全局表示。推理路径引入不一致性感知机制,结合语义原型匹配与统计概率校准以精确量化冲突严重程度,并动态调整两条路径间的权重。此外,采用多视角损失函数缓解模态惰性,并在不同阶段学习结构化特征。在两个基准数据集上的大量实验表明,CDPR在缓解多模态不一致性方面达到了最先进的性能与卓越的鲁棒性。代码已开源至https://github.com/Hebust-NLP/CDPR。