Recent advances in 3D content creation mostly leverage optimization-based 3D generation via score distillation sampling (SDS). Though promising results have been exhibited, these methods often suffer from slow per-sample optimization, limiting their practical usage. In this paper, we propose DreamGaussian, a novel 3D content generation framework that achieves both efficiency and quality simultaneously. Our key insight is to design a generative 3D Gaussian Splatting model with companioned mesh extraction and texture refinement in UV space. In contrast to the occupancy pruning used in Neural Radiance Fields, we demonstrate that the progressive densification of 3D Gaussians converges significantly faster for 3D generative tasks. To further enhance the texture quality and facilitate downstream applications, we introduce an efficient algorithm to convert 3D Gaussians into textured meshes and apply a fine-tuning stage to refine the details. Extensive experiments demonstrate the superior efficiency and competitive generation quality of our proposed approach. Notably, DreamGaussian produces high-quality textured meshes in just 2 minutes from a single-view image, achieving approximately 10 times acceleration compared to existing methods.
翻译:近期三维内容创作领域的研究主要基于分数蒸馏采样(SDS)的优化式三维生成方法。尽管这些方法已展现出可喜成果,但单样本优化速度缓慢的缺陷限制了其实际应用。本文提出DreamGaussian这一新型三维内容生成框架,在保持高质量输出的同时实现高效生成。我们的核心思路是构建生成式三维高斯泼溅模型,并配套设计UV空间中的网格提取与纹理优化模块。与神经辐射场中使用的占用剪枝策略不同,我们证明三维高斯逐步致密化方法在三维生成任务中具有显著更快的收敛速度。为提升纹理质量并便于下游应用,我们引入高效算法将三维高斯体转换为带纹理网格,并通过微调阶段优化细节呈现。大量实验表明,本方法在生成效率与质量上均具优势。值得注意的是,DreamGaussian仅需2分钟即可从单视角图像生成高质量带纹理网格,相较现有方法实现约10倍加速。