Reconstructing a 3D scene from images is challenging due to the different ways light interacts with surfaces depending on the viewer's position and the surface's material. In classical computer graphics, materials can be classified as diffuse or specular, interacting with light differently. The standard 3D Gaussian Splatting model struggles to represent view-dependent content, since it cannot differentiate an object within the scene from the light interacting with its specular surfaces, which produce highlights or reflections. In this paper, we propose to extend the 3D Gaussian Splatting model by introducing an additional symmetric matrix to enhance the opacity representation of each 3D Gaussian. This improvement allows certain Gaussians to be suppressed based on the viewer's perspective, resulting in a more accurate representation of view-dependent reflections and specular highlights without compromising the scene's integrity. By allowing the opacity to be view dependent, our enhanced model achieves state-of-the-art performance on Mip-Nerf, Tanks\&Temples, Deep Blending, and Nerf-Synthetic datasets without a significant loss in rendering speed, achieving >60FPS, and only incurring a minimal increase in memory used.

翻译：从图像重建三维场景具有挑战性，原因在于光线与表面的相互作用方式会随观察者位置及表面材质的不同而变化。在经典计算机图形学中，材质可分为漫反射与镜面反射两类，它们与光线的相互作用方式各异。标准的三维高斯溅射模型难以有效表征视角依赖型内容，因为它无法区分场景中的物体本身与作用于其镜面反射表面的光线，后者会产生高光或反射。本文提出对三维高斯溅射模型进行扩展，通过为每个三维高斯引入一个额外的对称矩阵来增强其不透明度表征能力。这一改进使得特定高斯可根据观察者视角被抑制，从而在不破坏场景完整性的前提下，更精确地呈现视角依赖型反射与镜面高光效果。通过使不透明度具备视角依赖性，我们改进的模型在Mip-Nerf、Tanks&Temples、Deep Blending及Nerf-Synthetic数据集上实现了最先进的性能，且渲染速度未显著降低（达到>60帧/秒），内存占用仅轻微增加。