Recently, Neural Radiance Fields (NeRF) has exhibited significant success in novel view synthesis, surface reconstruction, etc. However, since no physical reflection is considered in its rendering pipeline, NeRF mistakes the reflection in the mirror as a separate virtual scene, leading to the inaccurate reconstruction of the mirror and multi-view inconsistent reflections in the mirror. In this paper, we present a novel neural rendering framework, named Mirror-NeRF, which is able to learn accurate geometry and reflection of the mirror and support various scene manipulation applications with mirrors, such as adding new objects or mirrors into the scene and synthesizing the reflections of these new objects in mirrors, controlling mirror roughness, etc. To achieve this goal, we propose a unified radiance field by introducing the reflection probability and tracing rays following the light transport model of Whitted Ray Tracing, and also develop several techniques to facilitate the learning process. Experiments and comparisons on both synthetic and real datasets demonstrate the superiority of our method. The code and supplementary material are available on the project webpage: https://zju3dv.github.io/Mirror-NeRF/.

翻译：最近，神经辐射场（NeRF）在新视角合成、表面重建等方面展现出了显著的成功。然而，由于在其渲染流程中未考虑物理反射，NeRF会将镜面中的反射误认为独立的虚拟场景，导致镜面重建不准确以及镜面中的多视角反射不一致。在本文中，我们提出了一种新颖的神经渲染框架，名为Mirror-NeRF，它能够学习镜面的精确几何与反射，并支持多种涉及镜面的场景操控应用，例如在场景中添加新物体或镜面、合成这些新物体在镜面中的反射、控制镜面粗糙度等。为实现这一目标，我们通过引入反射概率并遵循Whitted光线追踪的光传输模型追踪光线，提出了一种统一的辐射场，同时还开发了多种技术以促进学习过程。在合成数据集和真实数据集上的实验与比较展示了我们方法的优越性。代码和补充材料可在项目网页上获取：https://zju3dv.github.io/Mirror-NeRF/。