Recent years have seen a growing interest in Scene Graph Generation (SGG), a comprehensive visual scene understanding task that aims to predict entity relationships using a relation encoder-decoder pipeline stacked on top of an object encoder-decoder backbone. Unfortunately, current SGG methods suffer from an information loss regarding the entities local-level cues during the relation encoding process. To mitigate this, we introduce the Vision rElation TransfOrmer (VETO), consisting of a novel local-level entity relation encoder. We further observe that many existing SGG methods claim to be unbiased, but are still biased towards either head or tail classes. To overcome this bias, we introduce a Mutually Exclusive ExperT (MEET) learning strategy that captures important relation features without bias towards head or tail classes. Experimental results on the VG and GQA datasets demonstrate that VETO + MEET boosts the predictive performance by up to 47 percentage over the state of the art while being 10 times smaller.

翻译：近年来，场景图生成（SGG）作为一项综合性视觉场景理解任务受到广泛关注，其目标是在物体编码器-解码器主干网络上叠加关系编码器-解码器流水线，以预测实体间的关系。遗憾的是，当前SGG方法在关系编码过程中存在实体局部级线索的信息丢失问题。为解决这一局限，我们提出视觉关系Transformer（VETO），该模型包含一个新颖的局部级实体关系编码器。我们进一步观察到，许多现有SGG方法声称是无偏的，但实际仍对头类或尾类存在偏向。为克服这一偏差，我们引入互斥专家（MEET）学习策略，该策略能够在不偏向头类或尾类的情况下捕捉重要关系特征。在VG和GQA数据集上的实验结果表明，VETO+MEET在模型规模小10倍的情况下，将预测性能相较于现有最优方法提升了高达47个百分点。