We focus on the task of soundscape mapping, which involves predicting the most probable sounds that could be perceived at a particular geographic location. We utilise recent state-of-the-art models to encode geotagged audio, a textual description of the audio, and an overhead image of its capture location using contrastive pre-training. The end result is a shared embedding space for the three modalities, which enables the construction of soundscape maps for any geographic region from textual or audio queries. Using the SoundingEarth dataset, we find that our approach significantly outperforms the existing SOTA, with an improvement of image-to-audio Recall@100 from 0.256 to 0.450. Our code is available at https://github.com/mvrl/geoclap.

翻译：我们聚焦于声景映射任务，该任务旨在预测特定地理位置最可能被感知到的声音。我们利用最新最优模型，通过对比预训练对带有地理标签的音频、音频的文本描述以及其采集地点的俯视图像进行编码。最终成果是为三种模态构建共享嵌入空间，从而能够根据文本或音频查询为任何地理区域创建声景地图。基于SoundingEarth数据集，我们发现该方法显著超越现有最优技术，将图像到音频的Recall@100从0.256提升至0.450。我们的代码已开源：https://github.com/mvrl/geoclap。