Existing work on Multimodal Sentiment Analysis (MSA) utilizes multimodal information for prediction yet unavoidably suffers from fitting the spurious correlations between multimodal features and sentiment labels. For example, if most videos with a blue background have positive labels in a dataset, the model will rely on such correlations for prediction, while "blue background" is not a sentiment-related feature. To address this problem, we define a general debiasing MSA task, which aims to enhance the Out-Of-Distribution (OOD) generalization ability of MSA models by reducing their reliance on spurious correlations. To this end, we propose a general debiasing framework based on Inverse Probability Weighting (IPW), which adaptively assigns small weights to the samples with larger bias (i.e., the severer spurious correlations). The key to this debiasing framework is to estimate the bias of each sample, which is achieved by two steps: 1) disentangling the robust features and biased features in each modality, and 2) utilizing the biased features to estimate the bias. Finally, we employ IPW to reduce the effects of large-biased samples, facilitating robust feature learning for sentiment prediction. To examine the model's generalization ability, we keep the original testing sets on two benchmarks and additionally construct multiple unimodal and multimodal OOD testing sets. The empirical results demonstrate the superior generalization ability of our proposed framework. We have released the code and data to facilitate the reproduction https://github.com/Teng-Sun/GEAR.

翻译：现有关于多模态情感分析（MSA）的研究利用多模态信息进行预测，但不可避免地会因拟合多模态特征与情感标签之间的虚假相关性而受到影响。例如，若数据集中大部分蓝色背景的视频带有正向标签，模型会依赖此类相关性进行预测，而“蓝色背景”并非情感相关特征。为解决此问题，我们定义了一个通用去偏MSA任务，旨在通过降低模型对虚假相关性的依赖，增强其分布外（OOD）泛化能力。为此，我们提出基于逆概率加权（IPW）的通用去偏框架，该框架自适应地为偏置较大（即虚假相关性更严重）的样本分配较小权重。该去偏框架的核心在于估计每个样本的偏置，具体通过两个步骤实现：1）分离每个模态中的鲁棒特征和偏置特征；2）利用偏置特征估计偏置。最后，我们采用逆概率加权降低高偏置样本的影响，从而促进情感预测的鲁棒特征学习。为检验模型的泛化能力，我们在两个基准测试中保留原始测试集，并额外构建多个单模态与多模态OOD测试集。实验结果表明，我们提出的框架具有优越的泛化能力。我们已公开代码与数据以支持复现：https://github.com/Teng-Sun/GEAR。