The creation of photorealistic virtual worlds requires the accurate modeling of 3D surface geometry for a wide range of objects. For this, meshes are appealing since they 1) enable fast physics-based rendering with realistic material and lighting, 2) support physical simulation, and 3) are memory-efficient for modern graphics pipelines. Recent work on reconstructing and statistically modeling 3D shape, however, has critiqued meshes as being topologically inflexible. To capture a wide range of object shapes, any 3D representation must be able to model solid, watertight, shapes as well as thin, open, surfaces. Recent work has focused on the former, and methods for reconstructing open surfaces do not support fast reconstruction with material and lighting or unconditional generative modelling. Inspired by the observation that open surfaces can be seen as islands floating on watertight surfaces, we parameterize open surfaces by defining a manifold signed distance field on watertight templates. With this parameterization, we further develop a grid-based and differentiable representation that parameterizes both watertight and non-watertight meshes of arbitrary topology. Our new representation, called Ghost-on-the-Shell (G-Shell), enables two important applications: differentiable rasterization-based reconstruction from multiview images and generative modelling of non-watertight meshes. We empirically demonstrate that G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks, while also performing effectively for watertight meshes.

翻译：创建逼真的虚拟世界需要对各种物体的三维表面几何进行精确建模。为此，网格（meshes）具有吸引力，原因如下：1）支持基于物理的快速渲染，实现逼真的材质与光照效果；2）支持物理仿真；3）对现代图形管线而言内存效率高。然而，近期关于三维形状重建与统计建模的工作批评网格在拓扑上缺乏灵活性。为了捕捉多样化的物体形状，任何三维表示必须能够建模实体、水密形状以及薄片、开放表面。近期工作主要关注前者，而重建开放表面的方法不支持带材质与光照的快速重建或无条件的生成式建模。受开放表面可视为漂浮在水密表面上的岛屿这一观察启发，我们通过在水密模板上定义流形符号距离场来参数化开放表面。基于此参数化，我们进一步开发了一种基于网格的可微表示，能够参数化任意拓扑的水密与非水密网格。我们的新表示称为"壳中之灵"（Ghost-on-the-Shell, G-Shell），支持两项重要应用：基于可微光栅化的多视角图像重建以及非水密网格的生成式建模。实验证明，G-Shell在非水密网格重建与生成任务上达到了最先进性能，同时在水密网格上也表现有效。