The creation of photorealistic virtual worlds requires the accurate modeling of 3D surface geometry for a wide range of objects. For this, meshes are appealing since they 1) enable fast physics-based rendering with realistic material and lighting, 2) support physical simulation, and 3) are memory-efficient for modern graphics pipelines. Recent work on reconstructing and statistically modeling 3D shape, however, has critiqued meshes as being topologically inflexible. To capture a wide range of object shapes, any 3D representation must be able to model solid, watertight, shapes as well as thin, open, surfaces. Recent work has focused on the former, and methods for reconstructing open surfaces do not support fast reconstruction with material and lighting or unconditional generative modelling. Inspired by the observation that open surfaces can be seen as islands floating on watertight surfaces, we parameterize open surfaces by defining a manifold signed distance field on watertight templates. With this parameterization, we further develop a grid-based and differentiable representation that parameterizes both watertight and non-watertight meshes of arbitrary topology. Our new representation, called Ghost-on-the-Shell (G-Shell), enables two important applications: differentiable rasterization-based reconstruction from multiview images and generative modelling of non-watertight meshes. We empirically demonstrate that G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks, while also performing effectively for watertight meshes.

翻译：创建照片级逼真的虚拟世界需要对各类物体的三维表面几何进行精确建模。为此，网格（meshes）具有吸引力，因为它们能够：1）通过物理渲染实现具有真实材质与光照的快速渲染，2）支持物理模拟，3）在现代图形管线中具有内存高效性。然而，近期关于三维形状重建与统计建模的研究指出，网格在拓扑结构上缺乏灵活性。要捕捉广泛类别的物体形状，任何三维表示都必须能够建模实体、无孔的形状，以及薄壳、开放的表面。现有研究主要集中于前者，而用于重建开放表面的方法不支持带材质与光照的快速重建或无条件生成建模。受“开放表面可视为漂浮于无孔表面上的岛屿”这一观察启发，我们通过在无孔模板上定义流形有符号距离场来参数化开放表面。基于此参数化，我们进一步开发了一种基于网格的可微分表示，该表示能够参数化任意拓扑的封闭与非封闭网格。这种新表示被称为"壳之幻影"（Ghost-on-the-Shell, G-Shell），它支持两项重要应用：基于可微分光栅化的多视角图像重建，以及非封闭网格的生成建模。实验表明，G-Shell在非封闭网格重建与生成任务中达到了最先进性能，同时在封闭网格上同样表现优异。