The success of re-localisation has crucial implications for the practical deployment of robots operating within a prior map or relative to one another in real-world scenarios. Using single-modality, place recognition and localisation can be compromised in challenging environments such as forests. To address this, we propose a strategy to prevent lidar-based re-localisation failure using lidar-image cross-modality. Our solution relies on self-supervised 2D-3D feature matching to predict alignment and misalignment. Leveraging a deep network for lidar feature extraction and relative pose estimation between point clouds, we train a model to evaluate the estimated transformation. A model predicting the presence of misalignment is learned by analysing image-lidar similarity in the embedding space and the geometric constraints available within the region seen in both modalities in Euclidean space. Experimental results using real datasets (offline and online modes) demonstrate the effectiveness of the proposed pipeline for robust re-localisation in unstructured, natural environments.

翻译：重定位的成功对于机器人在真实场景中基于先验地图或相互之间进行实际部署具有关键意义。在森林等具有挑战性的环境中，使用单一模态的地点识别和定位可能受到影响。为解决这一问题，我们提出了一种利用激光雷达-图像跨模态来防止基于激光雷达的重定位失败的策略。我们的解决方案依赖于自监督的2D-3D特征匹配来预测对齐与不对齐。借助深度学习网络进行激光雷达特征提取以及点云之间的相对姿态估计，我们训练了一个模型来评估估计的变换。通过分析嵌入空间中的图像-激光雷达相似性以及欧几里得空间中两种模态均可见区域内的几何约束，学习了一个预测不对齐是否存在的模型。使用真实数据集（离线与在线模式）进行的实验结果证明了所提流水线在非结构化自然环境中实现鲁棒重定位的有效性。