The advent of neural and Gaussian-based radiance field methods have achieved great success in the field of novel view synthesis. However, specular reflection remains non-trivial, as the high frequency radiance field is notoriously difficult to fit stably and accurately. We present a deferred shading method to effectively render specular reflection with Gaussian splatting. The key challenge comes from the environment map reflection model, which requires accurate surface normal while simultaneously bottlenecks normal estimation with discontinuous gradients. We leverage the per-pixel reflection gradients generated by deferred shading to bridge the optimization process of neighboring Gaussians, allowing nearly correct normal estimations to gradually propagate and eventually spread over all reflective objects. Our method significantly outperforms state-of-the-art techniques and concurrent work in synthesizing high-quality specular reflection effects, demonstrating a consistent improvement of peak signal-to-noise ratio (PSNR) for both synthetic and real-world scenes, while running at a frame rate almost identical to vanilla Gaussian splatting.
翻译:基于神经和基于高斯的辐射场方法的出现在新视角合成领域取得了巨大成功。然而,镜面反射仍然具有挑战性,因为高频辐射场以难以稳定且精确拟合而著称。我们提出了一种延迟着色方法,利用高斯泼溅有效渲染镜面反射。关键挑战来自环境图反射模型,该模型需要精确的表面法线,同时因不连续的梯度而瓶颈法线估计。我们利用延迟着色生成的逐像素反射梯度来桥接相邻高斯的优化过程,使得近似正确的法线估计能够逐渐传播并最终覆盖所有反射物体。我们的方法在合成高质量镜面反射效果方面显著优于现有最先进技术及同期工作,在合成和真实场景中均展现出峰值信噪比的一致提升,同时运行帧率几乎与普通高斯泼溅相同。