The creation of photorealistic virtual worlds requires the accurate modeling of 3D surface geometry for a wide range of objects. For this, meshes are appealing since they 1) enable fast physics-based rendering with realistic material and lighting, 2) support physical simulation, and 3) are memory-efficient for modern graphics pipelines. Recent work on reconstructing and statistically modeling 3D shape, however, has critiqued meshes as being topologically inflexible. To capture a wide range of object shapes, any 3D representation must be able to model solid, watertight, shapes as well as thin, open, surfaces. Recent work has focused on the former, and methods for reconstructing open surfaces do not support fast reconstruction with material and lighting or unconditional generative modelling. Inspired by the observation that open surfaces can be seen as islands floating on watertight surfaces, we parameterize open surfaces by defining a manifold signed distance field on watertight templates. With this parameterization, we further develop a grid-based and differentiable representation that parameterizes both watertight and non-watertight meshes of arbitrary topology. Our new representation, called Ghost-on-the-Shell (G-Shell), enables two important applications: differentiable rasterization-based reconstruction from multiview images and generative modelling of non-watertight meshes. We empirically demonstrate that G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks, while also performing effectively for watertight meshes.

翻译：构建逼真虚拟世界需要为各类物体精确建模三维表面几何。网格模型之所以具有吸引力，原因在于：1）支持基于物理的快速渲染以呈现真实材质与光照效果；2）可进行物理仿真模拟；3）对现代图形流水线具有内存高效性。然而，近期关于三维形状重建与统计建模的研究指出，网格模型在拓扑结构上缺乏灵活性。为捕捉多样化的物体形状，任何三维表示都必须能够同时建模实体水密形状与薄壁开放表面。现有研究多聚焦于前者，而开放表面重建方法既不支持带材质光照的快速重建，也无法实现无条件生成建模。受开放表面可视为水密表面上浮岛这一观察启发，我们通过在水密模板上定义流形有符号距离场来参数化开放表面。基于该参数化方法，我们进一步开发了可微分网格表示，能够参数化任意拓扑结构的水密与非水密网格。这种名为"壳中鬼影"（Ghost-on-the-Shell, G-Shell）的新表示方法实现了两项重要应用：基于可微分光栅化的多视角图像重建，以及非水密网格的生成建模。实验证明，G-Shell在非水密网格重建与生成任务上达到最优性能，同时对水密网格同样保持优异效果。