Neural Radiance Fields (NeRF) have quickly become the primary approach for 3D reconstruction and novel view synthesis in recent years due to their remarkable performance. Despite the huge interest in NeRF methods, a practical use case of NeRFs has largely been ignored; the exploration of the scene space modelled by a NeRF. In this paper, for the first time in the literature, we propose and formally define the scene exploration framework as the efficient discovery of NeRF model inputs (i.e. coordinates and viewing angles), using which one can render novel views that adhere to user-selected criteria. To remedy the lack of approaches addressing scene exploration, we first propose two baseline methods called Guided-Random Search (GRS) and Pose Interpolation-based Search (PIBS). We then cast scene exploration as an optimization problem, and propose the criteria-agnostic Evolution-Guided Pose Search (EGPS) for efficient exploration. We test all three approaches with various criteria (e.g. saliency maximization, image quality maximization, photo-composition quality improvement) and show that our EGPS performs more favourably than other baselines. We finally highlight key points and limitations, and outline directions for future research in scene exploration.
翻译:摘要:神经辐射场(NeRF)近年来凭借其卓越性能迅速成为三维重建和新视角合成的主流方法。尽管NeRF方法引发了广泛关注,但其实际应用场景——即对NeRF所建模的场景空间进行探索——在很大程度上仍被忽视。本文首次在文献中提出并正式定义场景探索框架:通过高效发现NeRF模型输入(即坐标与视角参数),据此渲染符合用户选定标准的新视角。为弥补相关方法的缺失,我们首先提出两种基准方法:引导式随机搜索(GRS)和基于姿态插值的搜索(PIBS)。进而将场景探索形式化为优化问题,提出与标准无关的演化引导姿态搜索(EGPS)方法以实现高效探索。我们使用多种标准(如显著性最大化、图像质量最大化、摄影构图质量提升)对所有三种方法进行测试,结果表明EGPS性能优于其他基准方法。最后,我们指出关键要点与局限性,并勾勒场景探索的未来研究方向。