Despite significant advances in large-scale text-to-image models, achieving hyper-realistic human image generation remains a desirable yet unsolved task. Existing models like Stable Diffusion and DALL-E 2 tend to generate human images with incoherent parts or unnatural poses. To tackle these challenges, our key insight is that human image is inherently structural over multiple granularities, from the coarse-level body skeleton to fine-grained spatial geometry. Therefore, capturing such correlations between the explicit appearance and latent structure in one model is essential to generate coherent and natural human images. To this end, we propose a unified framework, HyperHuman, that generates in-the-wild human images of high realism and diverse layouts. Specifically, 1) we first build a large-scale human-centric dataset, named HumanVerse, which consists of 340M images with comprehensive annotations like human pose, depth, and surface normal. 2) Next, we propose a Latent Structural Diffusion Model that simultaneously denoises the depth and surface normal along with the synthesized RGB image. Our model enforces the joint learning of image appearance, spatial relationship, and geometry in a unified network, where each branch in the model complements to each other with both structural awareness and textural richness. 3) Finally, to further boost the visual quality, we propose a Structure-Guided Refiner to compose the predicted conditions for more detailed generation of higher resolution. Extensive experiments demonstrate that our framework yields the state-of-the-art performance, generating hyper-realistic human images under diverse scenarios. Project Page: https://snap-research.github.io/HyperHuman/

翻译：尽管大规模文本到图像模型取得了显著进展，实现超写实人体图像生成仍是一项理想但尚未解决的任务。现有模型如Stable Diffusion和DALL-E 2倾向于生成身体部位不协调或姿态不自然的人体图像。为解决这些挑战，我们的关键洞察在于：人体图像本质上具有多粒度结构特性，从粗粒度的人体骨架到细粒度的空间几何信息。因此，在单一模型中捕捉显式外观与潜结构之间的此类关联，对于生成连贯自然的人体图像至关重要。为此，我们提出统一框架HyperHuman，用于生成高真实感且布局多样化的野外人体图像。具体而言：1）我们首先构建大规模人体中心数据集HumanVerse，包含3.4亿张图像及人体姿态、深度图、表面法向等全面标注；2）其次提出潜结构扩散模型，在合成RGB图像的同时对深度图和表面法向进行去噪。该模型在统一网络中强制执行图像外观、空间关系与几何结构的联合学习，各分支通过结构感知与纹理丰富性相互补充；3）最后为进一步提升视觉质量，提出结构引导精修器，利用预测条件生成更高分辨率的细致图像。大量实验证明，本框架性能达到业界领先水平，可在多样场景下生成超写实人体图像。项目页面：https://snap-research.github.io/HyperHuman/