Neural Radiance Fields (NeRF) have quickly become the primary approach for 3D reconstruction and novel view synthesis in recent years due to their remarkable performance. Despite the huge interest in NeRF methods, a practical use case of NeRFs has largely been ignored; the exploration of the scene space modelled by a NeRF. In this paper, for the first time in the literature, we propose and formally define the scene exploration framework as the efficient discovery of NeRF model inputs (i.e. coordinates and viewing angles), using which one can render novel views that adhere to user-selected criteria. To remedy the lack of approaches addressing scene exploration, we first propose two baseline methods called Guided-Random Search (GRS) and Pose Interpolation-based Search (PIBS). We then cast scene exploration as an optimization problem, and propose the criteria-agnostic Evolution-Guided Pose Search (EGPS) for efficient exploration. We test all three approaches with various criteria (e.g. saliency maximization, image quality maximization, photo-composition quality improvement) and show that our EGPS performs more favourably than other baselines. We finally highlight key points and limitations, and outline directions for future research in scene exploration.
翻译:摘要:神经辐射场(Neural Radiance Fields, NeRF)近年来因其卓越性能,迅速成为三维重建与新视角合成的主要方法。尽管对NeRF方法的研究兴趣浓厚,但NeRF的一个实际应用场景——即对NeRF建模的场景空间进行探索——在很大程度上被忽视了。本文首次在文献中提出并正式定义了场景探索框架,将其视为高效发现NeRF模型输入(即坐标与视角),从而能够渲染符合用户选定标准的新视图的过程。为弥补场景探索方法的缺失,我们首先提出两种基线方法:引导随机搜索(Guided-Random Search, GRS)和位姿插值搜索(Pose Interpolation-based Search, PIBS)。随后,我们将场景探索建模为一个优化问题,并提出与标准无关的进化引导位姿搜索(Evolution-Guided Pose Search, EGPS)方法以实现高效探索。我们针对不同标准(如显著性最大化、图像质量最大化、照片构图质量提升)测试了所有三种方法,结果表明,我们的EGPS方法性能优于其他基线方法。最后,我们总结了关键点与局限性,并指出了场景探索领域未来的研究方向。