We present DreamAvatar, a text-and-shape guided framework for generating high-quality 3D human avatars with controllable poses. While encouraging results have been produced by recent methods on text-guided 3D common object generation, generating high-quality human avatars remains an open challenge due to the complexity of the human body's shape, pose, and appearance. We propose DreamAvatar to tackle this challenge, which utilizes a trainable NeRF for predicting density and color features for 3D points and a pre-trained text-to-image diffusion model for providing 2D self-supervision. Specifically, we leverage SMPL models to provide rough pose and shape guidance for the generation. We introduce a dual space design that comprises a canonical space and an observation space, which are related by a learnable deformation field through the NeRF, allowing for the transfer of well-optimized texture and geometry from the canonical space to the target posed avatar. Additionally, we exploit a normal-consistency regularization to allow for more vivid generation with detailed geometry and texture. Through extensive evaluations, we demonstrate that DreamAvatar significantly outperforms existing methods, establishing a new state-of-the-art for text-and-shape guided 3D human generation.

翻译：我们提出DreamAvatar，一种基于文本与形状引导的框架，用于生成具有可控姿态的高质量三维人体化身。尽管近期方法在文本引导的三维通用物体生成方面取得了令人鼓舞的成果，但由于人体形状、姿态和外观的复杂性，生成高质量的人体化身仍是一项开放性挑战。为应对这一挑战，我们提出DreamAvatar，该框架利用可训练神经辐射场预测三维点的密度与颜色特征，并借助预训练的文本到图像扩散模型提供二维自监督信号。具体而言，我们利用SMPL模型提供粗略的姿态与形状引导。通过引入由规范空间与观测空间构成的双空间设计，并借助神经辐射场中的可学习形变场建立两者关联，从而将规范空间中充分优化的纹理与几何迁移至目标姿态化身。此外，我们利用法向一致性正则化实现包含精细几何与纹理的更具生动画面的生成。大量评估表明，DreamAvatar显著优于现有方法，为文本与形状引导的三维人体生成建立了新的最优基准。