Visual-inertial localization is a key problem in computer vision and robotics applications such as virtual reality, self-driving cars, and aerial vehicles. The goal is to estimate an accurate pose of an object when either the environment or the dynamics are known. Absolute pose regression (APR) techniques directly regress the absolute pose from an image input in a known scene using convolutional and spatio-temporal networks. Odometry methods perform relative pose regression (RPR) that predicts the relative pose from a known object dynamic (visual or inertial inputs). The localization task can be improved by retrieving information from both data sources for a cross-modal setup, which is a challenging problem due to contradictory tasks. In this work, we conduct a benchmark to evaluate deep multimodal fusion based on pose graph optimization and attention networks. Auxiliary and Bayesian learning are utilized for the APR task. We show accuracy improvements for the APR-RPR task and for the RPR-RPR task for aerial vehicles and hand-held devices. We conduct experiments on the EuRoC MAV and PennCOSYVIO datasets and record and evaluate a novel industry dataset.

翻译：视觉-惯性定位是计算机视觉与机器人应用（如虚拟现实、自动驾驶车辆和飞行器）中的关键问题，其目标是在已知环境或动力学条件下估计物体的精确位姿。绝对位姿回归（APR）技术利用卷积网络和时空网络，从已知场景中的图像输入直接回归绝对位姿。里程计方法则通过相对位姿回归（RPR）从已知物体动力学（视觉或惯性输入）预测相对位姿。通过跨模态设置从两种数据源中检索信息可改进定位任务，但由于任务相互矛盾，这一过程极具挑战性。本文开展了一项基准评估，基于位姿图优化与注意力网络对深度多模态融合进行评价，并利用辅助学习与贝叶斯学习完成APR任务。我们验证了在飞行器与手持设备场景下，APR-RPR任务与RPR-RPR任务的精度提升。实验基于EuRoC MAV和PennCOSYVIO数据集开展，同时记录并评估了一个全新的工业数据集。