Visual-inertial localization is a key problem in computer vision and robotics applications such as virtual reality, self-driving cars, and aerial vehicles. The goal is to estimate an accurate pose of an object when either the environment or the dynamics are known. Absolute pose regression (APR) techniques directly regress the absolute pose from an image input in a known scene using convolutional and spatio-temporal networks. Odometry methods perform relative pose regression (RPR) that predicts the relative pose from a known object dynamic (visual or inertial inputs). The localization task can be improved by retrieving information from both data sources for a cross-modal setup, which is a challenging problem due to contradictory tasks. In this work, we conduct a benchmark to evaluate deep multimodal fusion based on pose graph optimization and attention networks. Auxiliary and Bayesian learning are utilized for the APR task. We show accuracy improvements for the APR-RPR task and for the RPR-RPR task for aerial vehicles and hand-held devices. We conduct experiments on the EuRoC MAV and PennCOSYVIO datasets and record and evaluate a novel industry dataset.

翻译：视觉-惯性定位是计算机视觉与机器人应用（如虚拟现实、自动驾驶车辆及飞行器）中的关键问题，其目标是在已知环境或动力学条件下准确估计物体的位姿。绝对位姿回归技术利用卷积网络和时空网络，从已知场景中的图像输入直接回归绝对位姿。里程计方法则通过相对位姿回归，根据已知物体动力学（视觉或惯性输入）预测相对位姿。通过从两种数据源中检索信息用于跨模态设置，可提升定位任务性能，然而这对矛盾任务构成了一个具有挑战性的问题。本研究基于位姿图优化和注意力网络，针对深度多模态融合开展了基准评测。针对绝对位姿回归任务，采用辅助学习与贝叶斯学习方法。我们展示了在飞行器与手持设备场景下，APR-RPR任务与RPR-RPR任务的精度提升。实验在EuRoC MAV与PennCOSYVIO数据集上进行，同时记录并评估了一个全新的工业数据集。