Recently, 3D Gaussian splatting (3D-GS) has achieved great success in reconstructing and rendering real-world scenes. To transfer the high rendering quality to generation tasks, a series of research works attempt to generate 3D-Gaussian assets from text. However, the generated assets have not achieved the same quality as those in reconstruction tasks. We observe that Gaussians tend to grow without control as the generation process may cause indeterminacy. Aiming at highly enhancing the generation quality, we propose a novel framework named GaussianDreamerPro. The main idea is to bind Gaussians to reasonable geometry, which evolves over the whole generation process. Along different stages of our framework, both the geometry and appearance can be enriched progressively. The final output asset is constructed with 3D Gaussians bound to mesh, which shows significantly enhanced details and quality compared with previous methods. Notably, the generated asset can also be seamlessly integrated into downstream manipulation pipelines, e.g. animation, composition, and simulation etc., greatly promoting its potential in wide applications. Demos are available at https://taoranyi.com/gaussiandreamerpro/.
翻译:近年来,三维高斯泼溅(3D-GS)技术在真实场景重建与渲染领域取得了巨大成功。为将高质量渲染能力迁移至生成任务,一系列研究工作尝试从文本生成三维高斯数字资产。然而,现有方法生成的资产质量尚未达到重建任务的水平。我们观察到,生成过程的不确定性会导致高斯分布无约束扩散。为显著提升生成质量,本文提出名为GaussianDreamerPro的新型框架。其核心思想是将高斯分布绑定至合理几何结构,该结构在生成过程中持续演化。通过框架各阶段的迭代,几何形态与外观特征均可实现渐进式优化。最终输出的数字资产由绑定至网格的三维高斯模型构成,与现有方法相比在细节表现与整体质量上均有显著提升。值得注意的是,生成资产可无缝集成至下游操控流程(如动画制作、场景合成、物理仿真等),极大拓展了其应用潜力。演示视频详见https://taoranyi.com/gaussiandreamerpro/。