Visual simultaneous localization and mapping (VSLAM) has broad applications, with state-of-the-art methods leveraging deep neural networks for better robustness and applicability. However, there is a lack of research in fusing these learning-based methods with multi-sensor information, which could be indispensable to push related applications to large-scale and complex scenarios. In this paper, we tightly integrate the trainable deep dense bundle adjustment (DBA) with multi-sensor information through a factor graph. In the framework, recurrent optical flow and DBA are performed among sequential images. The Hessian information derived from DBA is fed into a generic factor graph for multi-sensor fusion, which employs a sliding window and supports probabilistic marginalization. A pipeline for visual-inertial integration is firstly developed, which provides the minimum ability of metric-scale localization and mapping. Furthermore, other sensors (e.g., global navigation satellite system) are integrated for driftless and geo-referencing functionality. Extensive tests are conducted on both public datasets and self-collected datasets. The results validate the superior localization performance of our approach, which enables real-time dense mapping in large-scale environments. The code has been made open-source (https://github.com/GREAT-WHU/DBA-Fusion).

翻译：视觉同步定位与建图（VSLAM）具有广泛的应用，当前最先进的方法借助深度神经网络提升了鲁棒性和适用性。然而，将这类基于学习方法与多传感器信息进行融合的研究仍显不足，而这对于将相关应用推广至大规模复杂场景至关重要。本文通过因子图将可训练的深度密集束调整（DBA）与多传感器信息紧密集成。在该框架中，序列图像间执行循环光流和DBA操作，从DBA导出的海森信息被输入至通用因子图以进行多传感器融合，该因子图采用滑动窗口并支持概率边缘化。首先构建了视觉-惯性集成管线，提供度量尺度定位与建图的基础能力。进一步集成其他传感器（例如全球导航卫星系统）以实现无漂移和地理参考功能。我们在公开数据集和自采集数据集上开展了大量测试，结果验证了本方法优越的定位性能，并支持在大规模环境中进行实时密集建图。相关代码已开源（https://github.com/GREAT-WHU/DBA-Fusion）。