Despite the impressive performance of recent unbiased Scene Graph Generation (SGG) methods, the current debiasing literature mainly focuses on the long-tailed distribution problem, whereas it overlooks another source of bias, i.e., semantic confusion, which makes the SGG model prone to yield false predictions for similar relationships. In this paper, we explore a debiasing procedure for the SGG task leveraging causal inference. Our central insight is that the Sparse Mechanism Shift (SMS) in causality allows independent intervention on multiple biases, thereby potentially preserving head category performance while pursuing the prediction of high-informative tail relationships. However, the noisy datasets lead to unobserved confounders for the SGG task, and thus the constructed causal models are always causal-insufficient to benefit from SMS. To remedy this, we propose Two-stage Causal Modeling (TsCM) for the SGG task, which takes the long-tailed distribution and semantic confusion as confounders to the Structural Causal Model (SCM) and then decouples the causal intervention into two stages. The first stage is causal representation learning, where we use a novel Population Loss (P-Loss) to intervene in the semantic confusion confounder. The second stage introduces the Adaptive Logit Adjustment (AL-Adjustment) to eliminate the long-tailed distribution confounder to complete causal calibration learning. These two stages are model agnostic and thus can be used in any SGG model that seeks unbiased predictions. Comprehensive experiments conducted on the popular SGG backbones and benchmarks show that our TsCM can achieve state-of-the-art performance in terms of mean recall rate. Furthermore, TsCM can maintain a higher recall rate than other debiasing methods, which indicates that our method can achieve a better tradeoff between head and tail relationships.

翻译：尽管最近的无偏场景图生成（SGG）方法取得了令人瞩目的性能，当前的去偏文献主要关注长尾分布问题，却忽略了另一类偏差来源——语义混淆，这导致SGG模型容易对相似关系产生错误预测。本文探索了一种利用因果推断的SGG任务去偏流程。我们的核心见解在于：因果机制中的稀疏机制迁移（SMS）允许对多重偏差进行独立干预，从而在追求高信息量尾部关系预测的同时，有可能保持头部类别的性能。然而，嘈杂的数据集导致SGG任务存在未观测的混杂因素，因此构建的因果模型总是因果不充分的，难以从SMS中获益。为解决此问题，我们针对SGG任务提出了两阶段因果建模（TsCM），该方法将长尾分布和语义混淆作为结构因果模型（SCM）的混杂因素，然后将因果干预解耦为两个阶段。第一阶段是因果表征学习，其中我们使用新型群体损失（P-Loss）对语义混淆混杂因素进行干预。第二阶段引入自适应对数几率调整（AL-Adjustment），以消除长尾分布混杂因素，从而完成因果校准学习。这两个阶段是模型无关的，因此可应用于任何寻求无偏预测的SGG模型。在主流SGG骨干网络和基准数据集上进行的全面实验表明，我们的TsCM在平均召回率指标上达到了最先进的性能。此外，TsCM相比其他去偏方法能保持更高的召回率，这表明我们的方法能在头部和尾部关系之间实现更优的权衡。