Gaussian Splatting is a powerful tool for reconstructing diffuse scenes, but it struggles to simultaneously model specular reflections and the appearance of objects behind semi-transparent surfaces. These specular reflections and transmittance are essential for realistic novel view synthesis, and existing methods do not properly incorporate the underlying physical processes to simulate them. To address this issue, we propose RT-GS, a unified framework that integrates a microfacet material model and ray tracing to jointly model specular reflection and transmittance in Gaussian Splatting. We accomplish this by using separate Gaussian primitives for reflections and transmittance, which allow modeling distant reflections and reconstructing objects behind transparent surfaces concurrently. We utilize a differentiable ray tracing framework to obtain the specular reflection and transmittance appearance. Our experiments demonstrate that our method successfully produces reflections and recovers objects behind transparent surfaces in complex environments, achieving significant qualitative improvements over prior methods where these specular light interactions are prominent.

翻译：高斯点云渲染是重建漫反射场景的有力工具，但在同时建模镜面反射和半透明表面后的物体外观时存在困难。这些镜面反射与透射现象对于实现真实感的新视角合成至关重要，而现有方法未能恰当融入底层物理过程来模拟它们。为解决这一问题，我们提出RT-GS——一种统一框架，通过集成微面元材质模型与光线追踪，在高斯点云渲染中联合建模镜面反射与透射。我们采用分离的反射高斯基元和透射高斯基元，使模型能够同时处理远距离反射和透明表面后的物体重建。通过可微分光线追踪框架获取镜面反射与透射外观。实验结果表明，我们的方法能在复杂环境中成功生成反射效果并恢复透明表面后的物体，在镜面光交互显著的场景中相较于现有方法取得了质的提升。