The importance of building footprints and their inventory has been recognised as foundational spatial information for multiple societal problems. Extracting complex urban buildings involves the segmentation of very high-resolution (VHR) earth observation (EO) images. U-Net is a common deep learning network and foundation for its new incarnations like ResUnet, U-Net++ and U-Net3+ for such segmentation. The re-incarnations look for efficiency gain by re-designing the skip connection component and exploiting the multi-scale features in U-Net. However, skip connections do not always improve these networks and removing some of them provides efficiency gains and reduced network parameters. In this paper, we propose three dual skip connection mechanisms for U-Net, ResUnet, and U-Net3+. These mechanisms deepen the feature maps forwarded by the skip connections and allow us to study which skip connections need to be denser to yield the highest efficiency gain. The mechanisms are evaluated on feature maps of different scales in the three networks, producing nine new network configurations. The networks are evaluated against their original vanilla versions using four building footprint datasets (three existing and one new) of different spatial resolutions: VHR (0.3m), high-resolution (1m and 1.2m), and multi-resolution (0.3+0.6+1.2m). The proposed mechanisms report efficiency gain on four evaluation measures for U-Net and ResUnet, and up to 17.7% and 18.4% gain in F1 score and Intersection over Union (IoU) for U-Net3+. The codes will be available in a GitHub link after peer review.
翻译:建筑足迹及其清单的重要性已被视为解决多项社会问题的基础空间信息。复杂城市建筑的提取涉及对超高分辨率(VHR)地球观测(EO)图像的分割。U-Net是一种常见的深度学习网络,也是其新变体如ResUnet、U-Net++和U-Net3+进行此类分割的基础。这些变体通过重新设计跳连接组件并利用U-Net中的多尺度特征来寻求效率提升。然而,跳连接并非总能改善这些网络,移除部分跳连接反而能带来效率提升和网络参数减少。本文针对U-Net、ResUnet和U-Net3+提出了三种双跳连接机制。这些机制加深了由跳连接传递的特征图,并使我们能够研究哪些跳连接需要更密集以产生最高效率增益。这些机制在三种网络的不同尺度特征图上进行了评估,生成了九种新的网络配置。使用四个不同空间分辨率的建筑足迹数据集(三个现有数据集和一个新数据集):超高分辨率(0.3m)、高分辨率(1m和1.2m)以及多分辨率(0.3+0.6+1.2m),将所提网络与其原始基础版本进行了对比评估。所提机制在四项评估指标上报告了U-Net和ResUnet的效率提升,而U-Net3+在F1分数和交并比(IoU)上分别获得了最高17.7%和18.4%的提升。代码将在同行评审后通过GitHub链接提供。