As cities continue to burgeon, Urban Computing emerges as a pivotal discipline for sustainable development by harnessing the power of cross-domain data fusion from diverse sources (e.g., geographical, traffic, social media, and environmental data) and modalities (e.g., spatio-temporal, visual, and textual modalities). Recently, we are witnessing a rising trend that utilizes various deep-learning methods to facilitate cross-domain data fusion in smart cities. To this end, we propose the first survey that systematically reviews the latest advancements in deep learning-based data fusion methods tailored for urban computing. Specifically, we first delve into data perspective to comprehend the role of each modality and data source. Secondly, we classify the methodology into four primary categories: feature-based, alignment-based, contrast-based, and generation-based fusion methods. Thirdly, we further categorize multi-modal urban applications into seven types: urban planning, transportation, economy, public safety, society, environment, and energy. Compared with previous surveys, we focus more on the synergy of deep learning methods with urban computing applications. Furthermore, we shed light on the interplay between Large Language Models (LLMs) and urban computing, postulating future research directions that could revolutionize the field. We firmly believe that the taxonomy, progress, and prospects delineated in our survey stand poised to significantly enrich the research community. The summary of the comprehensive and up-to-date paper list can be found at https://github.com/yoshall/Awesome-Multimodal-Urban-Computing.
翻译:随着城市持续发展,城市计算通过融合来自不同来源(如地理、交通、社交媒体和环境数据)与模态(如时空、视觉和文本模态)的跨领域数据,成为推动可持续发展的关键学科。近年来,我们观察到利用各类深度学习方法促进智慧城市跨领域数据融合的显著趋势。为此,本文首次系统综述了面向城市计算的深度学习数据融合方法的最新进展。具体而言,首先从数据视角深入剖析各模态与数据源的作用;其次,将方法归纳为基于特征、对齐、对比和生成的四大类融合方法;再次,进一步将多模态城市应用划分为城市规划、交通、经济、公共安全、社会、环境和能源七种类型。相较于已有综述,本文更聚焦深度学习方法与城市计算应用之间的协同作用。此外,我们揭示了大型语言模型(LLMs)与城市计算之间的交互机制,提出了可能变革该领域的未来研究方向。我们坚信本文所阐述的分类体系、进展与前景将显著丰富学界研究。综合且及时更新的论文清单总结详见https://github.com/yoshall/Awesome-Multimodal-Urban-Computing。