We study the image-based geolocalization problem that aims to locate ground-view query images on cartographic maps. Previous methods often utilize cross-view localization techniques to match ground-view query images with 2D maps. However, the performance of these methods is frequently unsatisfactory due to the significant cross-view appearance differences. In this paper, we extend cross-view matching to 2.5D spaces, where the heights of the structures - such as trees, buildings, and other objects - can provide additional information to guide the cross-view matching. We present a new approach to learning representative embeddings from multi-model data. Specifically, we first align 2D maps to ground-view panoramic images with polar transform to reduce the gap between panoramic images and maps. Then we leverage global fusion to fuse the multi-modal features from 2D and 2.5D maps to increase the distinctiveness of location embeddings. We construct the first large-scale ground-to-2.5D map geolocalization dataset to validate our method and facilitate the research. We test our learned embeddings on two popular localization approaches, i.e., single-image based localization, and route based localization. Extensive experiments demonstrate that our proposed method achieves significantly higher localization accuracy and faster convergence than previous 2D map-based approaches.

翻译：我们研究基于图像的地理定位问题，旨在将地面视角的查询图像定位到制图地图上。以往方法常利用跨视角定位技术，将地面查询图像与二维地图进行匹配。然而，由于显著的跨视角外观差异，这些方法的性能往往不尽如人意。本文中，我们将跨视角匹配拓展到2.5维空间，其中树木、建筑物及其他物体等结构的高度信息能够为跨视角匹配提供额外指导。我们提出了一种从多模态数据中学习具有代表性嵌入的新方法。具体而言，我们首先通过极坐标变换将二维地图与地面全景图像对齐，以缩小全景图像与地图之间的差异。随后，利用全局融合技术融合来自二维和2.5维地图的多模态特征，以增强位置嵌入的区分性。我们构建了首个大规模地面-2.5维地图地理定位数据集，用于验证我们的方法并推动相关研究。我们在两种主流定位方法（即基于单张图像的定位和基于路径的定位）上测试了所学习的嵌入。大量实验表明，与以往基于二维地图的方法相比，我们提出的方法在定位精度和收敛速度上均有显著提升。