Recent developments in neural rendering techniques have greatly enhanced the rendering of photo-realistic 3D scenes across both academic and commercial fields. The latest method, known as 3D Gaussian Splatting (3D-GS), has set new benchmarks for rendering quality and speed. Nevertheless, the limitations of 3D-GS become pronounced in synthesizing new viewpoints, especially for views that greatly deviate from those seen during training. Additionally, issues such as dilation and aliasing arise when zooming in or out. These challenges can all be traced back to a single underlying issue: insufficient sampling. In our paper, we present a bootstrapping method that significantly addresses this problem. This approach employs a diffusion model to enhance the rendering of novel views using trained 3D-GS, thereby streamlining the training process. Our results indicate that bootstrapping effectively reduces artifacts, as well as clear enhancements on the evaluation metrics. Furthermore, we show that our method is versatile and can be easily integrated, allowing various 3D reconstruction projects to benefit from our approach.
翻译:近期神经渲染技术的发展极大提升了学术与商业领域中逼真三维场景的渲染效果。最新方法——3D高斯泼溅(3D-GS)——为渲染质量和速度设立了新基准。然而,3D-GS在合成新视角时的局限性变得显著,尤其是对于与训练视角偏差较大的视角。此外,缩放时会出现膨胀和锯齿等问题。这些挑战均可追溯至一个根本性问题:采样不足。本文提出一种引导方法,显著解决了这一问题。该方法利用扩散模型,通过训练后的3D-GS增强新视角的渲染,从而简化训练流程。结果表明,引导方法有效减少了伪影,并在评估指标上取得明显提升。此外,我们证明该方法具有普适性且易于集成,可使各类三维重建项目受益于我们的技术。