Dressed people reconstruction from images is a popular task with promising applications in the creative media and game industry. However, most existing methods reconstruct the human body and garments as a whole with the supervision of 3D models, which hinders the downstream interaction tasks and requires hard-to-obtain data. To address these issues, we propose an unsupervised separated 3D garments and human reconstruction model (USR), which reconstructs the human body and authentic textured clothes in layers without 3D models. More specifically, our method proposes a generalized surface-aware neural radiance field to learn the mapping between sparse multi-view images and geometries of the dressed people. Based on the full geometry, we introduce a Semantic and Confidence Guided Separation strategy (SCGS) to detect, segment, and reconstruct the clothes layer, leveraging the consistency between 2D semantic and 3D geometry. Moreover, we propose a Geometry Fine-tune Module to smooth edges. Extensive experiments on our dataset show that comparing with state-of-the-art methods, USR achieves improvements on both geometry and appearance reconstruction while supporting generalizing to unseen people in real time. Besides, we also introduce SMPL-D model to show the benefit of the separated modeling of clothes and the human body that allows swapping clothes and virtual try-on.

翻译：从图像中重建着装人体是一项具有广阔应用前景的热门任务，尤其在创意媒体与游戏行业。然而，现有方法大多在三维模型监督下将人体与服装作为整体进行重建，这阻碍了下游交互任务的发展，且需要难以获取的数据。为解决这些问题，我们提出了一种无监督的三维服装与人体分离重建模型（USR），该模型无需三维模型即可分层重建人体与具备真实纹理的服装。具体而言，我们的方法提出了一种广义曲面感知神经辐射场，以学习稀疏多视角图像与着装人体几何之间的映射。基于完整几何，我们引入了一种语义与置信度引导的分离策略（SCGS），利用二维语义与三维几何的一致性来检测、分割并重建服装层。此外，我们提出了一种几何微调模块用于平滑边缘。在我们数据集上的大量实验表明，与现有方法相比，USR在几何与外观重建方面均实现了提升，同时支持实时泛化至未见人物。此外，我们还引入了SMPL-D模型，展示了服装与人体分离建模的优势，可实现服装交换与虚拟试穿。