The recent advancements in 3D Gaussian splatting (3D-GS) have not only facilitated real-time rendering through modern GPU rasterization pipelines but have also attained state-of-the-art rendering quality. Nevertheless, despite its exceptional rendering quality and performance on standard datasets, 3D-GS frequently encounters difficulties in accurately modeling specular and anisotropic components. This issue stems from the limited ability of spherical harmonics (SH) to represent high-frequency information. To overcome this challenge, we introduce Spec-Gaussian, an approach that utilizes an anisotropic spherical Gaussian (ASG) appearance field instead of SH for modeling the view-dependent appearance of each 3D Gaussian. Additionally, we have developed a coarse-to-fine training strategy to improve learning efficiency and eliminate floaters caused by overfitting in real-world scenes. Our experimental results demonstrate that our method surpasses existing approaches in terms of rendering quality. Thanks to ASG, we have significantly improved the ability of 3D-GS to model scenes with specular and anisotropic components without increasing the number of 3D Gaussians. This improvement extends the applicability of 3D GS to handle intricate scenarios with specular and anisotropic surfaces. Project page is https://ingra14m.github.io/Spec-Gaussian-website/.

翻译：近期，3D高斯泼溅（3D-GS）技术的进展不仅通过现代GPU光栅化管线实现了实时渲染，还达到了最先进的渲染质量。然而，尽管在标准数据集上展现出卓越的渲染质量与性能，3D-GS在精确建模镜面反射和各向异性分量时仍常遇困难。此问题源于球谐函数（SH）表征高频信息的能力有限。为克服这一挑战，我们提出了Spec-Gaussian方法，该方法采用各向异性球面高斯（ASG）外观场替代SH，以建模每个3D高斯的视点依赖外观。此外，我们开发了一种由粗到精的训练策略，以提升学习效率并消除真实场景中因过拟合产生的漂浮伪影。实验结果表明，本方法在渲染质量上超越了现有技术。得益于ASG的引入，我们在不增加3D高斯数量的前提下，显著提升了3D-GS对具有镜面反射和各向异性分量场景的建模能力。这一改进拓展了3D-GS处理具有复杂镜面反射与各向异性表面场景的适用性。项目页面详见 https://ingra14m.github.io/Spec-Gaussian-website/。