This position paper argues for the use of \emph{structured generative models} (SGMs) for scene understanding. This requires the reconstruction of a 3D scene from an input image, whereby the contents of the image are causally explained in terms of models of instantiated objects, each with their own type, shape, appearance and pose, along with global variables like scene lighting and camera parameters. This approach also requires scene models which account for the co-occurrences and inter-relationships of objects in a scene. The SGM approach has the merits that it is compositional and generative, which lead to interpretability. To pursue the SGM agenda, we need models for objects and scenes, and approaches to carry out inference. We first review models for objects, which include ``things'' (object categories that have a well defined shape), and ``stuff'' (categories which have amorphous spatial extent). We then move on to review \emph{scene models} which describe the inter-relationships of objects. Perhaps the most challenging problem for SGMs is \emph{inference} of the objects, lighting and camera parameters, and scene inter-relationships from input consisting of a single or multiple images. We conclude with a discussion of issues that need addressing to advance the SGM agenda.

翻译：本文立场论文主张使用*结构化生成模型*（SGMs）进行场景理解。这要求从输入图像重建三维场景，其中图像内容通过实例化对象的模型进行因果解释，每个对象具有自身的类型、形状、外观和姿态，以及场景光照和相机参数等全局变量。该方法还需要能够解释场景中对象共现与相互关系的场景模型。结构化生成模型方法的优势在于其组合性和生成性，这带来了可解释性。为推进结构化生成模型研究议程，我们需要为对象和场景建立模型，并开发推理方法。我们首先回顾对象模型，包括"事物"（具有明确形状的对象类别）和"材料"（空间范围不定形的类别）。随后综述描述对象间相互关系的*场景模型*。对结构化生成模型而言，最具挑战性的问题可能是从单个或多个图像输入中*推理*对象、光照和相机参数以及场景相互关系。最后，我们讨论推进结构化生成模型研究议程所需解决的问题。