Reconstructing a 3D scene from images is challenging due to the different ways light interacts with surfaces depending on the viewer's position and the surface's material. In classical computer graphics, materials can be classified as diffuse or specular, interacting with light differently. The standard 3D Gaussian Splatting model struggles to represent view-dependent content, since it cannot differentiate an object within the scene from the light interacting with its specular surfaces, which produce highlights or reflections. In this paper, we propose to extend the 3D Gaussian Splatting model by introducing an additional symmetric matrix to enhance the opacity representation of each 3D Gaussian. This improvement allows certain Gaussians to be suppressed based on the viewer's perspective, resulting in a more accurate representation of view-dependent reflections and specular highlights without compromising the scene's integrity. By allowing the opacity to be view dependent, our enhanced model achieves state-of-the-art performance on Mip-Nerf, Tanks&Temples, Deep Blending, and Nerf-Synthetic datasets without a significant loss in rendering speed, achieving >60FPS, and only incurring a minimal increase in memory used.

翻译：从图像重建三维场景具有挑战性，因为光线与表面的相互作用方式会随观察者位置和表面材质的不同而变化。在经典计算机图形学中，材质可分为漫反射和镜面反射，它们与光线的相互作用方式不同。标准的3D高斯溅射模型难以有效表示视点依赖的内容，因为它无法区分场景中的物体与照射其镜面表面（产生高光或反射）的光线。本文提出通过为每个3D高斯引入一个额外的对称矩阵来增强其不透明度表示，从而扩展3D高斯溅射模型。这一改进允许根据观察者视角抑制特定高斯，从而在不损害场景完整性的前提下，更精确地表示视点依赖的反射和镜面高光。通过使不透明度具备视点依赖性，我们增强的模型在Mip-Nerf、Tanks&Temples、Deep Blending和Nerf-Synthetic数据集上实现了最先进的性能，且渲染速度未显著下降（>60FPS），内存占用仅轻微增加。