With dense inputs, Neural Radiance Fields (NeRF) is able to render photo-realistic novel views under static conditions. Although the synthesis quality is excellent, existing NeRF-based methods fail to obtain moderate three-dimensional (3D) structures. The novel view synthesis quality drops dramatically given sparse input due to the implicitly reconstructed inaccurate 3D-scene structure. We propose SfMNeRF, a method to better synthesize novel views as well as reconstruct the 3D-scene geometry. SfMNeRF leverages the knowledge from the self-supervised depth estimation methods to constrain the 3D-scene geometry during view synthesis training. Specifically, SfMNeRF employs the epipolar, photometric consistency, depth smoothness, and position-of-matches constraints to explicitly reconstruct the 3D-scene structure. Through these explicit constraints and the implicit constraint from NeRF, our method improves the view synthesis as well as the 3D-scene geometry performance of NeRF at the same time. In addition, SfMNeRF synthesizes novel sub-pixels in which the ground truth is obtained by image interpolation. This strategy enables SfMNeRF to include more samples to improve generalization performance. Experiments on two public datasets demonstrate that SfMNeRF surpasses state-of-the-art approaches. Code is available at https://github.com/XTU-PR-LAB/SfMNeRF

翻译：针对密集输入，神经辐射场（NeRF）能够在静态条件下渲染出照片级真实感的新颖视图。尽管合成质量出色，但现有基于NeRF的方法难以获得合理的三维（3D）结构。由于隐式重建的不准确3D场景结构，在给定稀疏输入时，新颖视图合成质量会急剧下降。我们提出SfMNeRF，一种能同时优化新颖视图合成与3D场景几何重建的方法。SfMNeRF利用自监督深度估计方法的知识，在视图合成训练过程中约束3D场景几何。具体而言，SfMNeRF采用极线约束、光度一致性约束、深度平滑性约束和匹配位置约束，显式地重建3D场景结构。通过这些显式约束以及NeRF的隐式约束，我们的方法同时提升了NeRF的视图合成与3D场景几何性能。此外，SfMNeRF合成新颖子像素，其真实值通过图像插值获得。该策略使SfMNeRF能够包含更多样本以提高泛化性能。在两个公开数据集上的实验表明，SfMNeRF超越了当前最先进的方法。代码地址：https://github.com/XTU-PR-LAB/SfMNeRF