Implicit representations such as Neural Radiance Fields (NeRF) have been shown to be very effective at novel view synthesis. However, these models typically require manual and careful human data collection for training. In this paper, we present AutoNeRF, a method to collect data required to train NeRFs using autonomous embodied agents. Our method allows an agent to explore an unseen environment efficiently and use the experience to build an implicit map representation autonomously. We compare the impact of different exploration strategies including handcrafted frontier-based exploration, end-to-end and modular approaches composed of trained high-level planners and classical low-level path followers. We train these models with different reward functions tailored to this problem and evaluate the quality of the learned representations on four different downstream tasks: classical viewpoint rendering, map reconstruction, planning, and pose refinement. Empirical results show that NeRFs can be trained on actively collected data using just a single episode of experience in an unseen environment, and can be used for several downstream robotic tasks, and that modular trained exploration models outperform other classical and end-to-end baselines. Finally, we show that AutoNeRF can reconstruct large-scale scenes, and is thus a useful tool to perform scene-specific adaptation as the produced 3D environment models can be loaded into a simulator to fine-tune a policy of interest.

翻译：隐式表示方法（如神经辐射场）在新视角合成任务中展现出显著效果。然而，这类模型的训练通常需要人工精心采集数据。本文提出AutoNeRF——一种利用自主具身智能体收集数据以训练神经辐射场的方法。该方法使得智能体能够高效探索未知环境，并自主利用经验构建隐式地图表示。我们对比了不同探索策略的影响，包括人工设计的基于前沿的探索方法、端到端方法，以及由训练好的高层规划器与经典低层路径跟踪器组成的模块化方法。针对该问题，我们采用定制化的奖励函数训练这些模型，并在四项下游任务中评估学习到的表示质量：经典视角渲染、地图重建、规划与位姿优化。实验结果表明：在未知环境中仅需单轮经验积累即可基于主动采集数据训练神经辐射场，且该模型可用于多项下游机器人任务；同时，模块化训练探索模型优于其他经典方法与端到端基线。最后，我们证明AutoNeRF能够重建大规模场景，由于生成的三维环境模型可加载至仿真器中用于微调目标策略，因此该工具在场景自适应领域具有重要应用价值。