It is highly desirable to obtain a model that can generate high-quality 3D meshes from text prompts in just seconds. While recent attempts have adapted pre-trained text-to-image diffusion models, such as Stable Diffusion (SD), into generators of 3D representations (e.g., Triplane), they often suffer from poor quality due to the lack of sufficient high-quality 3D training data. Aiming at overcoming the data shortage, we propose a novel training scheme, termed as Progressive Rendering Distillation (PRD), eliminating the need for 3D ground-truths by distilling multi-view diffusion models and adapting SD into a native 3D generator. In each iteration of training, PRD uses the U-Net to progressively denoise the latent from random noise for a few steps, and in each step it decodes the denoised latent into 3D output. Multi-view diffusion models, including MVDream and RichDreamer, are used in joint with SD to distill text-consistent textures and geometries into the 3D outputs through score distillation. Since PRD supports training without 3D ground-truths, we can easily scale up the training data and improve generation quality for challenging text prompts with creative concepts. Meanwhile, PRD can accelerate the inference speed of the generation model in just a few steps. With PRD, we train a Triplane generator, namely TriplaneTurbo, which adds only $2.5\%$ trainable parameters to adapt SD for Triplane generation. TriplaneTurbo outperforms previous text-to-3D generators in both efficiency and quality. Specifically, it can produce high-quality 3D meshes in 1.2 seconds and generalize well for challenging text input. The code is available at https://github.com/theEricMa/TriplaneTurbo.

翻译：从文本提示快速生成高质量三维网格的模型具有重要应用价值。现有研究尝试将预训练的文本到图像扩散模型（如Stable Diffusion）适配为三维表征（如三平面）生成器，但由于缺乏充足的高质量三维训练数据，生成质量往往欠佳。为克服数据短缺问题，我们提出名为渐进式渲染蒸馏（PRD）的新型训练方案，通过蒸馏多视角扩散模型将SD适配为原生三维生成器，无需三维真值数据。在每次训练迭代中，PRD利用U-Net对随机噪声的隐变量进行渐进式去噪，并在每一步将去噪后的隐变量解码为三维输出。通过联合使用MVDream和RichDreamer等多视角扩散模型与SD，借助分数蒸馏将文本一致的纹理与几何特征注入三维输出。由于PRD支持无三维真值训练，我们能够轻松扩展训练数据规模，提升具有创造性概念的挑战性文本提示的生成质量。同时，PRD可将生成模型的推理速度加速至仅需数步。基于PRD方案，我们训练了名为TriplaneTurbo的三平面生成器，该模型仅增加2.5%可训练参数即可将SD适配为三平面生成器。TriplaneTurbo在效率与质量上均超越现有文本到三维生成器：仅需1.2秒即可生成高质量三维网格，并对挑战性文本输入展现出优异泛化能力。代码发布于https://github.com/theEricMa/TriplaneTurbo。