We present a two-stage text-to-3D generation system, namely 3DTopia, which generates high-quality general 3D assets within 5 minutes using hybrid diffusion priors. The first stage samples from a 3D diffusion prior directly learned from 3D data. Specifically, it is powered by a text-conditioned tri-plane latent diffusion model, which quickly generates coarse 3D samples for fast prototyping. The second stage utilizes 2D diffusion priors to further refine the texture of coarse 3D models from the first stage. The refinement consists of both latent and pixel space optimization for high-quality texture generation. To facilitate the training of the proposed system, we clean and caption the largest open-source 3D dataset, Objaverse, by combining the power of vision language models and large language models. Experiment results are reported qualitatively and quantitatively to show the performance of the proposed system. Our codes and models are available at https://github.com/3DTopia/3DTopia

翻译：本文提出一个两阶段文本到3D生成系统，即3DTopia，该系统利用混合扩散先验在5分钟内生成高质量通用3D资产。第一阶段从直接基于3D数据学习的3D扩散先验中采样。具体而言，该阶段由文本条件化的三平面潜扩散模型驱动，可快速生成粗粒度3D样本以实现快速原型构建。第二阶段利用2D扩散先验进一步优化第一阶段粗粒度3D模型的纹理。该优化过程结合潜空间和像素空间优化，以实现高质量纹理生成。为促进所提系统的训练，我们通过联合视觉语言模型与大语言模型的能力，对最大开源3D数据集Objaverse进行清洗与标注。实验结果的定性与定量分析展示了所提系统的性能。我们的代码和模型已开源至https://github.com/3DTopia/3DTopia。