Radiance fields have demonstrated impressive performance in synthesizing novel views from sparse input views, yet prevailing methods suffer from high training costs and slow inference speed. This paper introduces DNGaussian, a depth-regularized framework based on 3D Gaussian radiance fields, offering real-time and high-quality few-shot novel view synthesis at low costs. Our motivation stems from the highly efficient representation and surprising quality of the recent 3D Gaussian Splatting, despite it will encounter a geometry degradation when input views decrease. In the Gaussian radiance fields, we find this degradation in scene geometry primarily lined to the positioning of Gaussian primitives and can be mitigated by depth constraint. Consequently, we propose a Hard and Soft Depth Regularization to restore accurate scene geometry under coarse monocular depth supervision while maintaining a fine-grained color appearance. To further refine detailed geometry reshaping, we introduce Global-Local Depth Normalization, enhancing the focus on small local depth changes. Extensive experiments on LLFF, DTU, and Blender datasets demonstrate that DNGaussian outperforms state-of-the-art methods, achieving comparable or better results with significantly reduced memory cost, a $25 \times$ reduction in training time, and over $3000 \times$ faster rendering speed.
翻译:辐射场在从稀疏输入视角合成新视角方面展现了显著性能,但现有方法存在训练成本高、推理速度慢的问题。本文提出DNGaussian——一种基于三维高斯辐射场的深度正则化框架,能够以低成本实现实时且高质量的少样本新视角合成。我们的动机源于近期三维高斯溅射技术的高效表示能力和令人惊奇的合成质量,然而当输入视角减少时会出现几何退化。在高斯辐射场中,我们发现在场景几何中这种退化主要与高斯原语的定位有关,并且可以通过深度约束加以缓解。因此,我们提出硬深度正则化和软深度正则化,在保持细腻颜色外观的同时,基于粗略的单目深度监督恢复精确的场景几何。为进一步细化几何重塑细节,我们引入全局-局部深度归一化,增强对小范围局部深度变化的关注。在LLFF、DTU和Blender数据集上的大量实验表明,DNGaussian优于现有最先进方法,在显著降低内存成本的同时取得相当或更优的结果,训练时间缩减25倍,渲染速度提升超过3000倍。