Dynamic Neural Radiance Field (NeRF) is a powerful algorithm capable of rendering photo-realistic novel view images from a monocular RGB video of a dynamic scene. Although it warps moving points across frames from the observation spaces to a common canonical space for rendering, dynamic NeRF does not model the change of the reflected color during the warping. As a result, this approach often fails drastically on challenging specular objects in motion. We address this limitation by reformulating the neural radiance field function to be conditioned on surface position and orientation in the observation space. This allows the specular surface at different poses to keep the different reflected colors when mapped to the common canonical space. Additionally, we add the mask of moving objects to guide the deformation field. As the specular surface changes color during motion, the mask mitigates the problem of failure to find temporal correspondences with only RGB supervision. We evaluate our model based on the novel view synthesis quality with a self-collected dataset of different moving specular objects in realistic environments. The experimental results demonstrate that our method significantly improves the reconstruction quality of moving specular objects from monocular RGB videos compared to the existing NeRF models. Our code and data are available at the project website https://github.com/JokerYan/NeRF-DS.

翻译：动态神经辐射场（Dynamic NeRF）是一种强大的算法，能够从动态场景的单目RGB视频中渲染出逼真的新颖视角图像。尽管该算法通过将运动点从观测空间扭曲到公共的规范空间以进行渲染，但动态NeRF并未建模扭曲过程中反射颜色的变化。因此，该方法在处理运动的挑战性镜面物体时往往表现不佳。为解决这一局限，我们将神经辐射场函数重新构建为观测空间中表面位置和朝向的条件函数。这使得不同姿态下的镜面表面在映射到公共规范空间时能够保留不同的反射颜色。此外，我们添加了运动物体的掩码以引导形变场。由于镜面表面在运动过程中颜色会发生变化，该掩码缓解了仅通过RGB监督无法找到时间对应关系的问题。我们基于在真实环境中自采集的多种运动镜面物体数据集，评估了模型在新视角合成质量上的表现。实验结果表明，与现有NeRF模型相比，我们的方法显著提升了从单目RGB视频中重建运动镜面物体的质量。我们的代码和数据可在项目网站 https://github.com/JokerYan/NeRF-DS 上获取。