The recent advancements in image-text diffusion models have stimulated research interest in large-scale 3D generative models. Nevertheless, the limited availability of diverse 3D resources presents significant challenges to learning. In this paper, we present a novel method for generating high-quality, stylized 3D avatars that utilizes pre-trained image-text diffusion models for data generation and a Generative Adversarial Network (GAN)-based 3D generation network for training. Our method leverages the comprehensive priors of appearance and geometry offered by image-text diffusion models to generate multi-view images of avatars in various styles. During data generation, we employ poses extracted from existing 3D models to guide the generation of multi-view images. To address the misalignment between poses and images in data, we investigate view-specific prompts and develop a coarse-to-fine discriminator for GAN training. We also delve into attribute-related prompts to increase the diversity of the generated avatars. Additionally, we develop a latent diffusion model within the style space of StyleGAN to enable the generation of avatars based on image inputs. Our approach demonstrates superior performance over current state-of-the-art methods in terms of visual quality and diversity of the produced avatars.

翻译：近期图文扩散模型的进展激发了对大规模三维生成模型的研究兴趣。然而，三维资源多样性的匮乏给学习过程带来了显著挑战。本文提出一种新颖的高质量风格化三维虚拟人生成方法，该方法利用预训练图文扩散模型生成数据，并采用基于生成对抗网络（GAN）的三维生成网络进行训练。我们的方法借助图文扩散模型提供的丰富外观与几何先验，生成多种风格的虚拟人多视角图像。在数据生成阶段，我们使用从现有三维模型中提取的姿态来引导多视角图像的生成。为解决数据中姿态与图像不对齐的问题，我们研究了视角特定提示，并为GAN训练开发了粗到细判别器。我们还深入探究了属性相关提示以增强生成虚拟人的多样性。此外，我们在StyleGAN风格空间中构建潜在扩散模型，实现基于图像输入的虚拟人生成。我们的方法在生成虚拟人的视觉质量和多样性方面均优于当前最先进方法。