We present SpectroMotion, a novel approach that combines 3D Gaussian Splatting (3DGS) with physically-based rendering (PBR) and deformation fields to reconstruct dynamic specular scenes. Previous methods extending 3DGS to model dynamic scenes have struggled to represent specular surfaces accurately. Our method addresses this limitation by introducing a residual correction technique for accurate surface normal computation during deformation, complemented by a deformable environment map that adapts to time-varying lighting conditions. We implement a coarse-to-fine training strategy significantly enhancing scene geometry and specular color prediction. It is the only existing 3DGS method capable of synthesizing photorealistic real-world dynamic specular scenes, outperforming state-of-the-art methods in rendering complex, dynamic, and specular scenes.

翻译：我们提出SpectroMotion，一种将三维高斯泼溅（3DGS）与基于物理的渲染（PBR）及变形场相结合以重建动态镜面场景的新方法。先前将3DGS扩展至动态场景建模的方法难以精确表示镜面表面。我们的方法通过引入变形过程中精确计算表面法向的残差校正技术，并结合可适应时变光照条件的可变形环境贴图，解决了这一局限性。我们实现了由粗到精的训练策略，显著提升了场景几何与镜面颜色预测的准确性。这是目前唯一能够合成逼真现实世界动态镜面场景的3DGS方法，在渲染复杂、动态且具有镜面特性的场景方面超越了现有最先进方法。