This paper proposes a novel multi-temporal urban mapping approach using multi-modal satellite data from the Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 MultiSpectral Instrument (MSI) missions. In particular, it focuses on the problem of a partly missing optical modality due to clouds. The proposed model utilizes two networks to extract features from each modality separately. In addition, a reconstruction network is utilized to approximate the optical features based on the SAR data in case of a missing optical modality. Our experiments on a multi-temporal urban mapping dataset with Sentinel-1 SAR and Sentinel-2 MSI data demonstrate that the proposed method outperforms a multi-modal approach that uses zero values as a replacement for missing optical data, as well as a uni-modal SAR-based approach. Therefore, the proposed method is effective in exploiting multi-modal data, if available, but it also retains its effectiveness in case the optical modality is missing.

翻译：本文提出了一种新颖的多时间城市测绘方法，利用来自哨兵1号合成孔径雷达（Sentinel-1 SAR）和哨兵2号多光谱成像仪（Sentinel-2 MSI）任务的多模态卫星数据。特别地，该方法聚焦于因云层导致光学模态部分缺失的问题。所提出的模型采用两个网络分别从每种模态中提取特征。此外，当光学模态缺失时，利用重构网络基于SAR数据逼近光学特征。我们在包含哨兵1号SAR和哨兵2号MSI数据的多时间城市测绘数据集上的实验表明，所提出的方法优于使用零值替代缺失光学数据的多模态方法，以及基于SAR的单模态方法。因此，该方法在可利用多模态数据时能有效挖掘其价值，同时在光学模态缺失时仍能保持有效性。