We present a novel deep learning-based approach to the 3D reconstruction of clothed humans using weak supervision via 2D normal maps. Given a single RGB image or multiview images, our network infers a signed distance function (SDF) discretized on a tetrahedral mesh surrounding the body in a rest pose. Subsequently, inferred pose and camera parameters are used to generate a normal map from the SDF. A key aspect of our approach is the use of Marching Tetrahedra to (uniquely) compute a triangulated surface from the SDF on the tetrahedral mesh, facilitating straightforward differentiation (and thus backpropagation). Thus, given only ground truth normal maps (with no volumetric information ground truth information), we can train the network to produce SDF values from corresponding RGB images. Optionally, an additional multiview loss leads to improved results. We demonstrate the efficacy of our approach for both network inference and 3D reconstruction.

翻译：我们提出一种基于深度学习的新方法，通过二维法线图的弱监督信号实现三维穿衣人体的重建。给定单张RGB图像或多视角图像，我们的网络在静止姿态下人体周围的四面体网格上推断离散化的符号距离函数。随后，利用推断的姿态与相机参数，从该符号距离函数生成法线图。该方法的核心创新在于采用移动四面体算法，从四面体网格上的符号距离函数唯一地计算三角化表面，从而实现便捷的微分运算（进而支持反向传播）。因此，仅需真实法线图作为监督（无需任何体积信息真值），即可训练网络从对应RGB图像中预测符号距离函数值。此外，引入多视角损失函数可进一步优化重建效果。实验证明，本方法在网络推理与三维重建任务中均展现出卓越性能。