Dressed people reconstruction from images is a popular task with promising applications in the creative media and game industry. However, most existing methods reconstruct the human body and garments as a whole with the supervision of 3D models, which hinders the downstream interaction tasks and requires hard-to-obtain data. To address these issues, we propose an unsupervised separated 3D garments and human reconstruction model (USR), which reconstructs the human body and authentic textured clothes in layers without 3D models. More specifically, our method proposes a generalized surface-aware neural radiance field to learn the mapping between sparse multi-view images and geometries of the dressed people. Based on the full geometry, we introduce a Semantic and Confidence Guided Separation strategy (SCGS) to detect, segment, and reconstruct the clothes layer, leveraging the consistency between 2D semantic and 3D geometry. Moreover, we propose a Geometry Fine-tune Module to smooth edges. Extensive experiments on our dataset show that comparing with state-of-the-art methods, USR achieves improvements on both geometry and appearance reconstruction while supporting generalizing to unseen people in real time. Besides, we also introduce SMPL-D model to show the benefit of the separated modeling of clothes and the human body that allows swapping clothes and virtual try-on.

翻译：从图像中重建穿衣人物是创意媒体与游戏行业中具有广阔应用前景的热门任务。然而，现有方法大多在三维模型监督下将人体与服装作为整体进行重建，这阻碍了下游交互任务，且需要难以获取的数据。为解决这些问题，我们提出了一种无监督的三维服装与人体分离重建模型（USR），该模型无需三维模型即可分层重建人体与逼真纹理服装。具体而言，我们的方法提出了一种广义曲面感知神经辐射场，用于学习稀疏多视角图像与穿衣人物几何结构之间的映射。基于完整几何结构，我们引入语义与置信度引导分离策略（SCGS），利用二维语义与三维几何之间的一致性，实现服装层的检测、分割与重建。此外，我们提出几何微调模块以平滑边缘。在我们数据集上的大量实验表明，与现有最优方法相比，USR在几何与外观重建方面均取得提升，同时支持对未见人物进行实时泛化。此外，我们还引入了SMPL-D模型，以展示服装与人体分离建模的优势——支持服装交换与虚拟试穿。