In recent times, the generation of 3D assets from text prompts has shown impressive results. Both 2D and 3D diffusion models can help generate decent 3D objects based on prompts. 3D diffusion models have good 3D consistency, but their quality and generalization are limited as trainable 3D data is expensive and hard to obtain. 2D diffusion models enjoy strong abilities of generalization and fine generation, but 3D consistency is hard to guarantee. This paper attempts to bridge the power from the two types of diffusion models via the recent explicit and efficient 3D Gaussian splatting representation. A fast 3D object generation framework, named as GaussianDreamer, is proposed, where the 3D diffusion model provides priors for initialization and the 2D diffusion model enriches the geometry and appearance. Operations of noisy point growing and color perturbation are introduced to enhance the initialized Gaussians. Our GaussianDreamer can generate a high-quality 3D instance or 3D avatar within 15 minutes on one GPU, much faster than previous methods, while the generated instances can be directly rendered in real time. Demos and code are available at https://taoranyi.com/gaussiandreamer/.

翻译：近期，从文本提示生成3D资产的研究取得了显著成果。2D与3D扩散模型均能基于提示生成可用的3D对象。3D扩散模型具有良好的3D一致性，但由于可训练的3D数据成本高昂且难以获取，其生成质量与泛化能力受限。2D扩散模型具备强大的泛化能力与精细生成能力，但难以保证3D一致性。本文尝试通过近期提出的显式高效3D高斯泼溅表示，桥接两类扩散模型的优势。我们提出名为GaussianDreamer的快速3D对象生成框架，其中3D扩散模型为初始化提供先验，2D扩散模型则丰富几何与外观信息。引入噪声点生长与颜色扰动操作以增强初始化的高斯体。GaussianDreamer可在单GPU上15分钟内生成高质量3D实例或3D化身，速度远超此前方法，且生成的实例可直接实时渲染。演示与代码详见https://taoranyi.com/gaussiandreamer/。