Most existing debiasing methods for multimodal models, including causal intervention and inference methods, utilize approximate heuristics to represent the biases, such as shallow features from early stages of training or unimodal features for multimodal tasks like VQA, etc., which may not be accurate. In this paper, we study bias arising from confounders in a causal graph for multimodal data and examine a novel approach that leverages causally-motivated information minimization to learn the confounder representations. Robust predictive features contain diverse information that helps a model generalize to out-of-distribution data. Hence, minimizing the information content of features obtained from a pretrained biased model helps learn the simplest predictive features that capture the underlying data distribution. We treat these features as confounder representations and use them via methods motivated by causal theory to remove bias from models. We find that the learned confounder representations indeed capture dataset biases, and the proposed debiasing methods improve out-of-distribution (OOD) performance on multiple multimodal datasets without sacrificing in-distribution performance. Additionally, we introduce a novel metric to quantify the sufficiency of spurious features in models' predictions that further demonstrates the effectiveness of our proposed methods. Our code is available at: https://github.com/Vaidehi99/CausalInfoMin

翻译：现有的多模态模型去偏方法（包括因果干预与推理方法）大多采用近似启发式策略来表征偏差，例如利用训练早期阶段的浅层特征或面向VQA等任务提取单模态特征，但这些方法可能存在精度不足的问题。本文针对多模态数据因果图中的混杂因素引发的偏差展开研究，提出一种基于因果驱动的信息最小化学习混杂表征的新方法。鲁棒预测特征包含的多样化信息有助于模型泛化至分布外数据，因此通过最小化预训练偏置模型所提取特征的信息量，可学习到捕获底层数据分布的最简预测特征。我们将此类特征视为混杂表征，并基于因果理论驱动的方法将其用于消除模型偏差。实验表明：所学混杂表征确实能够捕获数据集偏差，所提去偏方法可在不牺牲分布内性能的前提下提升多模态数据集的分布外（OOD）性能。此外，我们提出了一种量化模型预测中虚假特征充分性的新指标，进一步验证了所提方法的有效性。代码开源地址：https://github.com/Vaidehi99/CausalInfoMin