Existing visual question reasoning methods usually fail to explicitly discover the inherent causal mechanism and ignore the complex event-level understanding that requires jointly modeling cross-modal event temporality and causality. In this paper, we propose an event-level visual question reasoning framework named Cross-Modal Question Reasoning (CMQR), to explicitly discover temporal causal structure and mitigate visual spurious correlation by causal intervention. To explicitly discover visual causal structure, the Visual Causality Discovery (VCD) architecture is proposed to find question-critical scene temporally and disentangle the visual spurious correlations by attention-based front-door causal intervention module named Local-Global Causal Attention Module (LGCAM). To align the fine-grained interactions between linguistic semantics and spatial-temporal representations, we build an Interactive Visual-Linguistic Transformer (IVLT) that builds the multi-modal co-occurrence interactions between visual and linguistic content. Extensive experiments on four datasets demonstrate the superiority of CMQR for discovering visual causal structures and achieving robust question reasoning.

翻译：现有的视觉问题推理方法通常无法明确发现内在的因果机制，且忽视了需要联合建模跨模态事件时序性与因果性的复杂事件级理解。本文提出一个名为跨模态问题推理（CMQR）的事件级视觉问题推理框架，通过因果干预显式发现时序因果结构并缓解视觉虚假相关性。为显式发现视觉因果结构，我们设计了视觉因果发现（VCD）架构，用于时序性地定位问题关键场景，并通过基于注意力机制的前门因果干预模块——局部-全局因果注意力模块（LGCAM）解耦视觉虚假相关性。为对齐语言语义与时空表征之间的细粒度交互，我们构建了交互式视觉-语言Transformer（IVLT），建立视觉与语言内容之间的多模态共现交互。在四个数据集上的大量实验表明，CMQR在发现视觉因果结构及实现鲁棒问题推理方面具有优越性。