Given a single in-the-wild human photo, it remains a challenging task to reconstruct a high-fidelity 3D human model. Existing methods face difficulties including a) the varying body proportions captured by in-the-wild human images; b) diverse personal belongings within the shot; and c) ambiguities in human postures and inconsistency in human textures. In addition, the scarcity of high-quality human data intensifies the challenge. To address these problems, we propose a Generalizable image-to-3D huMAN reconstruction framework, dubbed GeneMAN, building upon a comprehensive multi-source collection of high-quality human data, including 3D scans, multi-view videos, single photos, and our generated synthetic human data. GeneMAN encompasses three key modules. 1) Without relying on parametric human models (e.g., SMPL), GeneMAN first trains a human-specific text-to-image diffusion model and a view-conditioned diffusion model, serving as GeneMAN 2D human prior and 3D human prior for reconstruction, respectively. 2) With the help of the pretrained human prior models, the Geometry Initialization-&-Sculpting pipeline is leveraged to recover high-quality 3D human geometry given a single image. 3) To achieve high-fidelity 3D human textures, GeneMAN employs the Multi-Space Texture Refinement pipeline, consecutively refining textures in the latent and the pixel spaces. Extensive experimental results demonstrate that GeneMAN could generate high-quality 3D human models from a single image input, outperforming prior state-of-the-art methods. Notably, GeneMAN could reveal much better generalizability in dealing with in-the-wild images, often yielding high-quality 3D human models in natural poses with common items, regardless of the body proportions in the input images.

翻译：给定一张真实场景下的单人照片，重建高保真度的三维人体模型仍然是一项具有挑战性的任务。现有方法面临诸多困难，包括：a) 真实人体图像所捕捉到的多变人体比例；b) 拍摄画面中多样的个人物品；c) 人体姿态的模糊性以及人体纹理的不一致性。此外，高质量人体数据的稀缺性加剧了这一挑战。为解决这些问题，我们提出了一种可泛化的图像到三维人体重建框架，命名为 GeneMAN。该框架基于一个全面的多源高质量人体数据集合构建，包括三维扫描数据、多视角视频、单张照片以及我们生成的合成人体数据。GeneMAN 包含三个关键模块。1) 在不依赖参数化人体模型（例如 SMPL）的情况下，GeneMAN 首先训练了一个人体特定的文本到图像扩散模型和一个视角条件扩散模型，分别作为 GeneMAN 重建所需的二维人体先验和三维人体先验。2) 借助预训练的人体先验模型，利用几何初始化与雕刻流水线，从单张图像中恢复高质量的三维人体几何形状。3) 为实现高保真度的三维人体纹理，GeneMAN 采用了多空间纹理细化流水线，在潜空间和像素空间中连续细化纹理。大量的实验结果表明，GeneMAN 能够从单张图像输入生成高质量的三维人体模型，其性能优于先前的最先进方法。值得注意的是，GeneMAN 在处理真实场景图像时展现出更好的泛化能力，无论输入图像中的人体比例如何，通常都能生成具有自然姿态和常见物品的高质量三维人体模型。