Artificial intelligence (AI) approaches nowadays have gained remarkable success in single-modality-dominated remote sensing (RS) applications, especially with an emphasis on individual urban environments (e.g., single cities or regions). Yet these AI models tend to meet the performance bottleneck in the case studies across cities or regions, due to the lack of diverse RS information and cutting-edge solutions with high generalization ability. To this end, we build a new set of multimodal remote sensing benchmark datasets (including hyperspectral, multispectral, SAR) for the study purpose of the cross-city semantic segmentation task (called C2Seg dataset), which consists of two cross-city scenes, i.e., Berlin-Augsburg (in Germany) and Beijing-Wuhan (in China). Beyond the single city, we propose a high-resolution domain adaptation network, HighDAN for short, to promote the AI model's generalization ability from the multi-city environments. HighDAN is capable of retaining the spatially topological structure of the studied urban scene well in a parallel high-to-low resolution fusion fashion but also closing the gap derived from enormous differences of RS image representations between different cities by means of adversarial learning. In addition, the Dice loss is considered in HighDAN to alleviate the class imbalance issue caused by factors across cities. Extensive experiments conducted on the C2Seg dataset show the superiority of our HighDAN in terms of segmentation performance and generalization ability, compared to state-of-the-art competitors. The C2Seg dataset and the semantic segmentation toolbox (involving the proposed HighDAN) will be available publicly at https://github.com/danfenghong.

翻译：人工智能方法目前在单模态主导的遥感应用中取得了显著成功，尤其侧重于单一城市环境（如单个城市或区域）。然而，由于缺乏多样化的遥感信息以及具有高泛化能力的前沿解决方案，这些AI模型在跨城市或跨区域案例研究中往往遇到性能瓶颈。为此，我们构建了一套新型多模态遥感基准数据集（包括高光谱、多光谱、SAR），用于研究跨城市语义分割任务（简称C2Seg数据集），其中包含两个跨城市场景，即柏林-奥格斯堡（德国）和北京-武汉（中国）。针对单一城市之外的场景，我们提出了一种高分辨率域适应网络（简称HighDAN），以提升AI模型在多城市环境中的泛化能力。HighDAN不仅能通过并行的高低分辨率融合方式有效保留所研究城市场景的空间拓扑结构，还能借助对抗学习缩小不同城市间遥感图像表征巨大差异所带来的鸿沟。此外，HighDAN中引入了Dice损失，以缓解跨城市因素导致的类别不平衡问题。在C2Seg数据集上进行的大量实验表明，与最先进的竞争方法相比，我们的HighDAN在分割性能和泛化能力方面均具有优越性。C2Seg数据集及语义分割工具箱（包含所提出的HighDAN）将在https://github.com/danfenghong上公开提供。