This paper proposes a novel visual simultaneous localization and mapping (SLAM) system called Hybrid Depth-augmented Panoramic Visual SLAM (HDPV-SLAM), that employs a panoramic camera and a tilted multi-beam LiDAR scanner to generate accurate and metrically-scaled trajectories. RGB-D SLAM was the design basis for HDPV-SLAM, which added depth information to visual features. It aims to solve the two major issues hindering the performance of similar SLAM systems. The first obstacle is the sparseness of LiDAR depth, which makes it difficult to correlate it with the extracted visual features of the RGB image. A deep learning-based depth estimation module for iteratively densifying sparse LiDAR depth was suggested to address this issue. The second issue pertains to the difficulties in depth association caused by a lack of horizontal overlap between the panoramic camera and the tilted LiDAR sensor. To surmount this difficulty, we present a hybrid depth association module that optimally combines depth information estimated by two independent procedures, feature-based triangulation and depth estimation. During a phase of feature tracking, this hybrid depth association module aims to maximize the use of more accurate depth information between the triangulated depth with visual features tracked and the deep learning-based corrected depth. We evaluated the efficacy of HDPV-SLAM using the 18.95 km-long York University and Teledyne Optech (YUTO) MMS dataset. The experimental results demonstrate that the two proposed modules contribute substantially to the performance of HDPV-SLAM, which surpasses that of the state-of-the-art (SOTA) SLAM systems.

翻译：摘要：本文提出一种新型视觉同步定位与建图（SLAM）系统，称为混合深度增强全景视觉SLAM（HDPV-SLAM），该系统采用全景相机和倾斜多波束激光雷达扫描仪生成精确且具有公制尺度的轨迹。HDPV-SLAM以RGB-D SLAM为设计基础，为视觉特征添加深度信息，旨在解决阻碍同类SLAM系统性能的两个主要问题。第一个障碍是激光雷达深度的稀疏性，导致其难以与RGB图像中提取的视觉特征建立关联。为此，我们提出一种基于深度学习的深度估计模块，用于迭代式地对稀疏激光雷达深度进行稠密化处理。第二个问题涉及因全景相机与倾斜激光雷达传感器缺乏水平重叠区域而导致的深度关联困难。为克服这一难题，我们提出一种混合深度关联模块，该模块可优化结合由两种独立过程（基于特征的三角测量和深度估计）估计的深度信息。在特征跟踪阶段，该混合深度关联模块旨在最大化利用三角测量深度（与跟踪的视觉特征相关）与基于深度学习的校正深度之间更精确的深度信息。我们利用长18.95公里的约克大学与Teledyne Optech（YUTO）移动测绘系统数据集评估了HDPV-SLAM的性能。实验结果表明，所提出的两个模块对HDPV-SLAM的性能有显著贡献，其性能超越了最先进的SLAM系统。