We introduce TADA, a simple-yet-effective approach that takes textual descriptions and produces expressive 3D avatars with high-quality geometry and lifelike textures, that can be animated and rendered with traditional graphics pipelines. Existing text-based character generation methods are limited in terms of geometry and texture quality, and cannot be realistically animated due to inconsistent alignment between the geometry and the texture, particularly in the face region. To overcome these limitations, TADA leverages the synergy of a 2D diffusion model and an animatable parametric body model. Specifically, we derive an optimizable high-resolution body model from SMPL-X with 3D displacements and a texture map, and use hierarchical rendering with score distillation sampling (SDS) to create high-quality, detailed, holistic 3D avatars from text. To ensure alignment between the geometry and texture, we render normals and RGB images of the generated character and exploit their latent embeddings in the SDS training process. We further introduce various expression parameters to deform the generated character during training, ensuring that the semantics of our generated character remain consistent with the original SMPL-X model, resulting in an animatable character. Comprehensive evaluations demonstrate that TADA significantly surpasses existing approaches on both qualitative and quantitative measures. TADA enables creation of large-scale digital character assets that are ready for animation and rendering, while also being easily editable through natural language. The code will be public for research purposes.
翻译:我们提出TADA,一种简单而有效的方法,该方法接受文本描述,生成具有高质量几何和逼真纹理的可表达3D化身,这些化身可通过传统图形管线进行动画化和渲染。现有基于文本的角色生成方法在几何和纹理质量方面有限,且由于几何与纹理(尤其是面部区域)之间不一致的对应关系,无法实现逼真的动画化。为克服这些限制,TADA利用2D扩散模型与可动画化的参数化身体模型的协同作用。具体而言,我们从SMPL-X导出一个可优化的高分辨率身体模型,包含3D位移和纹理贴图,并使用分层渲染结合得分蒸馏采样(SDS)从文本创建高质量、精细、整体的3D化身。为确保几何与纹理之间的对齐,我们渲染生成角色的法线图和RGB图像,并在SDS训练过程中利用其潜在嵌入。我们进一步引入各种表情参数,在训练过程中对生成角色进行变形,确保生成角色的语义与原始SMPL-X模型保持一致,从而得到一个可动画化的角色。综合评估表明,TADA在定性和定量指标上均显著超越现有方法。TADA能够创建可直接用于动画化和渲染的大规模数字角色资产,同时可通过自然语言轻松编辑。代码将公开以供研究用途。