Generating adversarial scenarios, which have the potential to fail autonomous driving systems, provides an effective way to improve robustness. Extending purely data-driven generative models, recent specialized models satisfy additional controllable requirements such as embedding a traffic sign in a driving scene by manipulating patterns implicitly in the neuron level. In this paper, we introduce a method to incorporate domain knowledge explicitly in the generation process to achieve the Semantically Adversarial Generation (SAG). To be consistent with the composition of driving scenes, we first categorize the knowledge into two types, the property of objects and the relationship among objects. We then propose a tree-structured variational auto-encoder (T-VAE) to learn hierarchical scene representation. By imposing semantic rules on the properties of nodes and edges in the tree structure, explicit knowledge integration enables controllable generation. We construct a synthetic example to illustrate the controllability and explainability of our method in a succinct setting. We further extend to realistic environments for autonomous vehicles: our method efficiently identifies adversarial driving scenes against different state-of-the-art 3D point cloud segmentation models and satisfies the traffic rules specified as the explicit knowledge.

翻译：生成可能使自动驾驶系统失效的对抗性场景，为提高系统鲁棒性提供了有效途径。为扩展纯数据驱动的生成模型，近期专用模型通过神经元层面的隐式模式操控，满足了诸如在驾驶场景中嵌入交通标志等额外可控性需求。本文提出一种在生成过程中显式融入领域知识的方法，以实现语义对抗性生成（SAG）。为与驾驶场景的组成结构保持一致，我们首先将知识分为两类：物体的属性与物体间的关系。随后提出树结构变分自编码器（T-VAE）来学习层次化场景表征。通过在树结构的节点和边属性上施加语义规则，显式知识集成实现了可控生成。我们构建了一个合成示例，以简洁的设定阐明本方法的可控性与可解释性。进一步我们将该方法扩展到自动驾驶的真实环境中：本方法能够高效识别针对多种最先进的三维点云分割模型的对抗性驾驶场景，同时满足作为显式知识指定的交通规则。