The recent advancements in image-text diffusion models have stimulated research interest in large-scale 3D generative models. Nevertheless, the limited availability of diverse 3D resources presents significant challenges to learning. In this paper, we present a novel method for generating high-quality, stylized 3D avatars that utilizes pre-trained image-text diffusion models for data generation and a Generative Adversarial Network (GAN)-based 3D generation network for training. Our method leverages the comprehensive priors of appearance and geometry offered by image-text diffusion models to generate multi-view images of avatars in various styles. During data generation, we employ poses extracted from existing 3D models to guide the generation of multi-view images. To address the misalignment between poses and images in data, we investigate view-specific prompts and develop a coarse-to-fine discriminator for GAN training. We also delve into attribute-related prompts to increase the diversity of the generated avatars. Additionally, we develop a latent diffusion model within the style space of StyleGAN to enable the generation of avatars based on image inputs. Our approach demonstrates superior performance over current state-of-the-art methods in terms of visual quality and diversity of the produced avatars.

翻译：近年来图像-文本扩散模型的进展激发了学术界对大规模三维生成模型的研究兴趣。然而，多样化三维数据资源的匮乏给模型学习带来了重大挑战。本文提出了一种创新方法，通过利用预训练图像-文本扩散模型生成数据，并基于生成对抗网络的三维生成网络进行训练，从而生成高质量、风格化的三维虚拟形象。该方法借助图像-文本扩散模型提供的全面外观与几何先验知识，生成多视角、多风格的虚拟形象图像。在数据生成阶段，我们采用从现有三维模型中提取的姿态信息来引导多视角图像的生成。针对数据中姿态与图像的对齐偏差问题，我们研究了视角特定提示词，并开发了用于GAN训练的粗到细判别器。同时，我们深入探究了属性相关提示词以增强生成虚拟形象的多样性。此外，我们在StyleGAN风格空间中构建了潜在扩散模型，实现了基于图像输入的三维虚拟形象生成。实验表明，本方法在生成虚拟形象的视觉质量与多样性方面均显著优于现有最优方法。