We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task. The proposed graph-Transformer-based generator includes a novel graph Transformer encoder that combines graph convolutions and self-attentions in a Transformer to model both local and global interactions across connected and non-connected graph nodes. Specifically, the proposed connected node attention (CNA) and non-connected node attention (NNA) aim to capture the global relations across connected nodes and non-connected nodes in the input graph, respectively. The proposed graph modeling block (GMB) aims to exploit local vertex interactions based on a house layout topology. Moreover, we propose a new node classification-based discriminator to preserve the high-level semantic and discriminative node features for different house components. Finally, we propose a novel graph-based cycle-consistency loss that aims at maintaining the relative spatial relationships between ground truth and predicted graphs. Experiments on two challenging graph-constrained house generation tasks (i.e., house layout and roof generation) with two public datasets demonstrate the effectiveness of GTGAN in terms of objective quantitative scores and subjective visual realism. New state-of-the-art results are established by large margins on both tasks.

翻译：我们提出了一种新颖的图Transformer生成对抗网络（GTGAN），以端到端方式学习有效的图节点关系，用于具有挑战性的图约束房屋生成任务。所提出的基于图Transformer的生成器包含一个新颖的图Transformer编码器，该编码器在Transformer中结合了图卷积和自注意力机制，以建模连接和非连接图节点的局部和全局交互。具体而言，所提出的连接节点注意力（CNA）和非连接节点注意力（NNA）分别旨在捕获输入图中连接节点和非连接节点之间的全局关系。所提出的图建模模块（GMB）旨在基于房屋布局拓扑利用局部顶点交互。此外，我们提出了一种新的基于节点分类的鉴别器，以保留不同房屋组件的高层语义和判别性节点特征。最后，我们提出了一种新颖的基于图的循环一致性损失，旨在保持真实图和预测图之间的相对空间关系。在两个具有挑战性的图约束房屋生成任务（即房屋布局和屋顶生成）上的实验，以及两个公开数据集，证明了GTGAN在客观量化分数和主观视觉真实性方面的有效性。在这两项任务上，我们大幅建立了新的最先进结果。