Scene graph generation aims to construct a semantic graph structure from an image such that its nodes and edges respectively represent objects and their relationships. One of the major challenges for the task lies in the presence of distracting objects and relationships in images; contextual reasoning is strongly distracted by irrelevant objects or backgrounds and, more importantly, a vast number of irrelevant candidate relations. To tackle the issue, we propose the Selective Quad Attention Network (SQUAT) that learns to select relevant object pairs and disambiguate them via diverse contextual interactions. SQUAT consists of two main components: edge selection and quad attention. The edge selection module selects relevant object pairs, i.e., edges in the scene graph, which helps contextual reasoning, and the quad attention module then updates the edge features using both edge-to-node and edge-to-edge cross-attentions to capture contextual information between objects and object pairs. Experiments demonstrate the strong performance and robustness of SQUAT, achieving the state of the art on the Visual Genome and Open Images v6 benchmarks.

翻译：场景图生成旨在从图像中构建语义图结构，其中节点和边分别表示物体及其关系。该任务的主要挑战之一在于图像中存在干扰物体和关系；无关物体或背景，尤其是大量无关候选关系，会严重干扰上下文推理。为了解决这一问题，我们提出了选择性四元注意力网络（SQUAT），该网络能够学习选择相关物体对，并通过多样化的上下文交互对其进行消歧。SQUAT由两个主要组件构成：边选择和四元注意力。边选择模块选择相关物体对（即场景图中的边），有助于上下文推理；四元注意力模块随后通过边到节点和边到边的交叉注意力更新边特征，以捕获物体与物体对之间的上下文信息。实验证明了SQUAT的强大性能和鲁棒性，在Visual Genome和Open Images v6基准测试上达到了最先进的水平。