Recent advancements have focused on encoding urban spatial information into high-dimensional spaces, with notable efforts dedicated to integrating sociodemographic data and satellite imagery. These efforts have established foundational models in this field. However, the effective utilization of these spatial representations for urban forecasting applications remains under-explored. To address this gap, we introduce GeoTransformer, a novel structure that synergizes the Transformer architecture with geospatial statistics prior. GeoTransformer employs an innovative geospatial attention mechanism to incorporate extensive urban information and spatial dependencies into a unified predictive model. Specifically, we compute geospatial weighted attention scores between the target region and surrounding regions and leverage the integrated urban information for predictions. Extensive experiments on GDP and ride-share demand prediction tasks demonstrate that GeoTransformer significantly outperforms existing baseline models, showcasing its potential to enhance urban forecasting tasks.

翻译：近期研究进展主要集中于将城市空间信息编码至高维空间，其中将社会人口统计数据与卫星影像相融合的努力尤为显著。这些工作为该领域奠定了基础模型。然而，如何有效利用这些空间表征进行城市预测应用仍待深入探索。为填补这一空白，我们提出了GeoTransformer——一种将Transformer架构与地理空间统计先验知识相协同的新型结构。GeoTransformer采用创新的地理空间注意力机制，将广泛的城市信息与空间依赖关系整合到统一的预测模型中。具体而言，我们计算目标区域与周边区域之间的地理空间加权注意力分数，并利用整合后的城市信息进行预测。在GDP和共享出行需求预测任务上的大量实验表明，GeoTransformer显著优于现有基线模型，展现了其提升城市预测任务性能的潜力。