We present NeRFVS, a novel neural radiance fields (NeRF) based method to enable free navigation in a room. NeRF achieves impressive performance in rendering images for novel views similar to the input views while suffering for novel views that are significantly different from the training views. To address this issue, we utilize the holistic priors, including pseudo depth maps and view coverage information, from neural reconstruction to guide the learning of implicit neural representations of 3D indoor scenes. Concretely, an off-the-shelf neural reconstruction method is leveraged to generate a geometry scaffold. Then, two loss functions based on the holistic priors are proposed to improve the learning of NeRF: 1) A robust depth loss that can tolerate the error of the pseudo depth map to guide the geometry learning of NeRF; 2) A variance loss to regularize the variance of implicit neural representations to reduce the geometry and color ambiguity in the learning procedure. These two loss functions are modulated during NeRF optimization according to the view coverage information to reduce the negative influence brought by the view coverage imbalance. Extensive results demonstrate that our NeRFVS outperforms state-of-the-art view synthesis methods quantitatively and qualitatively on indoor scenes, achieving high-fidelity free navigation results.

翻译：我们提出NeRFVS，一种基于神经辐射场（NeRF）的新方法，用于实现房间内的自由导航。NeRF在渲染与输入视角相似的新视角图像方面取得了令人印象深刻的性能，但在处理与训练视角显著不同的新视角时会遇到困难。为解决这一问题，我们利用来自神经重建的整体先验信息，包括伪深度图和视角覆盖信息，来指导3D室内场景隐式神经表示的学习。具体而言，我们利用一个现成的神经重建方法生成几何支架。然后，基于整体先验信息提出两种损失函数以改进NeRF的学习：1）一种能够容忍伪深度图误差的鲁棒深度损失，用于指导NeRF的几何学习；2）一种方差损失，用于规范隐式神经表示的方差，以减少学习过程中几何和颜色的歧义性。根据视角覆盖信息对这两种损失函数在NeRF优化过程中进行调节，以减轻视角覆盖不平衡带来的负面影响。大量结果表明，我们的NeRFVS在室内场景的定量和定性评估中均优于最先进的新视角合成方法，实现了高保真的自由导航结果。