A prior map serves as a foundational reference for localization in context-aware applications such as augmented reality (AR). Providing valuable contextual information about the environment, the prior map is a vital tool for mitigating drift. In this paper, we propose a map-based visual-inertial localization algorithm (NeRF-VIO) with initialization using neural radiance fields (NeRF). Our algorithm utilizes a multilayer perceptron model and redefines the loss function as the geodesic distance on \(SE(3)\), ensuring the invariance of the initialization model under a frame change within \(\mathfrak{se}(3)\). The evaluation demonstrates that our model outperforms existing NeRF-based initialization solution in both accuracy and efficiency. By integrating a two-stage update mechanism within a multi-state constraint Kalman filter (MSCKF) framework, the state of NeRF-VIO is constrained by both captured images from an onboard camera and rendered images from a pre-trained NeRF model. The proposed algorithm is validated using a real-world AR dataset, the results indicate that our two-stage update pipeline outperforms MSCKF across all data sequences.

翻译：先验地图作为上下文感知应用（如增强现实）中定位的基础参考，提供了关于环境的有价值上下文信息，是抑制漂移的重要工具。本文提出了一种基于地图的视觉-惯性定位算法（NeRF-VIO），其初始化过程利用了神经辐射场（NeRF）。该算法采用多层感知机模型，并将损失函数重新定义为 \(SE(3)\) 上的测地距离，从而确保初始化模型在 \(\mathfrak{se}(3)\) 内的坐标系变换下具有不变性。评估结果表明，我们的模型在精度和效率上均优于现有的基于NeRF的初始化方案。通过在多状态约束卡尔曼滤波器（MSCKF）框架内集成两阶段更新机制，NeRF-VIO的状态同时受到机载摄像头捕获的图像和预训练NeRF模型渲染图像的约束。所提算法在真实世界增强现实数据集上进行了验证，结果表明我们的两阶段更新流程在所有数据序列上均优于MSCKF。