Recent developments in neural rendering techniques have greatly enhanced the rendering of photo-realistic 3D scenes across both academic and commercial fields. The latest method, known as 3D Gaussian Splatting (3D-GS), has set new benchmarks for rendering quality and speed. Nevertheless, the limitations of 3D-GS become pronounced in synthesizing new viewpoints, especially for views that greatly deviate from those seen during training. Additionally, issues such as dilation and aliasing arise when zooming in or out. These challenges can all be traced back to a single underlying issue: insufficient sampling. In our paper, we present a bootstrapping method that significantly addresses this problem. This approach employs a diffusion model to enhance the rendering of novel views using trained 3D-GS, thereby streamlining the training process. Our results indicate that bootstrapping effectively reduces artifacts, as well as clear enhancements on the evaluation metrics. Furthermore, we show that our method is versatile and can be easily integrated, allowing various 3D reconstruction projects to benefit from our approach.
翻译:近期神经渲染技术的发展显著提升了学术与商业领域中逼真三维场景的渲染效果。最新方法——3D高斯泼溅(3D-GS)在渲染质量与速度方面树立了新标杆。然而,在合成新视角(尤其是与训练视角偏差较大的视角)时,3D-GS的局限性变得尤为突出。此外,缩放操作还会引发膨胀与锯齿效应。这些挑战均可追溯至同一根本问题:采样不足。本文提出了一种引导方法,可有效解决该问题。该方法利用扩散模型通过已训练的3D-GS增强新视角渲染,从而简化训练流程。实验结果表明,引导方法能有效减少伪影,并在评估指标上取得显著提升。此外,我们证明该方法具备通用性与易集成性,可惠及各类三维重建项目。