NeRF aims to learn a continuous neural scene representation by using a finite set of input images taken from various viewpoints. A well-known limitation of NeRF methods is their reliance on data: the fewer the viewpoints, the higher the likelihood of overfitting. This paper addresses this issue by introducing a novel method to generate geometrically consistent image transitions between viewpoints using View Morphing. Our VM-NeRF approach requires no prior knowledge about the scene structure, as View Morphing is based on the fundamental principles of projective geometry. VM-NeRF tightly integrates this geometric view generation process during the training procedure of standard NeRF approaches. Notably, our method significantly improves novel view synthesis, particularly when only a few views are available. Experimental evaluation reveals consistent improvement over current methods that handle sparse viewpoints in NeRF models. We report an increase in PSNR of up to 1.8dB and 1.0dB when training uses eight and four views, respectively. Source code: \url{https://github.com/mbortolon97/VM-NeRF}
翻译:NeRF旨在通过使用从不同视点拍摄的有限输入图像集来学习连续的神经场景表示。NeRF方法的一个众所周知局限性是其对数据的依赖性:视点越少,过拟合的可能性越高。本文通过引入一种利用视角变形生成视点间几何一致图像过渡的新方法来解决这一问题。我们的VM-NeRF方法无需场景结构的先验知识,因为视角变形基于射影几何的基本原理。VM-NeRF将这一几何视角生成过程紧密集成到标准NeRF方法的训练流程中。值得注意的是,我们的方法显著提升了新视角合成质量,尤其是在仅有少量视点可用的情况下。实验评估表明,在处理NeRF模型中稀疏视点的现有方法上,我们的方法具有持续改进。当使用八个和四个视点进行训练时,我们报告的峰值信噪比分别提高至1.8dB和1.0dB。源代码:\url{https://github.com/mbortolon97/VM-NeRF}