We propose a novel type of map for visual navigation, a renderable neural radiance map (RNR-Map), which is designed to contain the overall visual information of a 3D environment. The RNR-Map has a grid form and consists of latent codes at each pixel. These latent codes are embedded from image observations, and can be converted to the neural radiance field which enables image rendering given a camera pose. The recorded latent codes implicitly contain visual information about the environment, which makes the RNR-Map visually descriptive. This visual information in RNR-Map can be a useful guideline for visual localization and navigation. We develop localization and navigation frameworks that can effectively utilize the RNR-Map. We evaluate the proposed frameworks on camera tracking, visual localization, and image-goal navigation. Experimental results show that the RNR-Map-based localization framework can find the target location based on a single query image with fast speed and competitive accuracy compared to other baselines. Also, this localization framework is robust to environmental changes, and even finds the most visually similar places when a query image from a different environment is given. The proposed navigation framework outperforms the existing image-goal navigation methods in difficult scenarios, under odometry and actuation noises. The navigation framework shows 65.7% success rate in curved scenarios of the NRNS dataset, which is an improvement of 18.6% over the current state-of-the-art. Project page: https://rllab-snu.github.io/projects/RNR-Map/

翻译：我们提出了一种新颖的视觉导航地图——可渲染神经辐射场地图（RNR-Map），该地图旨在包含3D环境的完整视觉信息。RNR-Map采用网格形式，每个像素存储潜编码。这些潜编码从图像观测中嵌入，并可转换为神经辐射场，从而支持根据相机位姿进行图像渲染。记录的潜编码隐式包含环境的视觉信息，使RNR-Map具备视觉描述能力。RNR-Map中的视觉信息可作为视觉定位与导航的有效指引。我们开发了能够高效利用RNR-Map的定位与导航框架，并在相机追踪、视觉定位及图像目标导航任务上评估了所提框架。实验结果表明，基于RNR-Map的定位框架能够通过单张查询图像快速且精确地定位目标位置，其性能与现有基线方法相比具有竞争力。此外，该定位框架对环境变化具有鲁棒性，甚至在给定不同环境的查询图像时，仍能定位到视觉上最相似的区域。所提出的导航框架在存在里程计与执行噪声的困难场景中优于现有图像目标导航方法。在NRNS数据集的弯曲场景中，该导航框架的成功率达到65.7%，较当前最先进方法提升了18.6%。项目页面：https://rllab-snu.github.io/projects/RNR-Map/