Self-supervised learning (SSL) has gained widespread attention in the remote sensing (RS) and earth observation (EO) communities owing to its ability to learn task-agnostic representations without human-annotated labels. Nevertheless, most existing RS SSL methods are limited to learning either global semantic separable or local spatial perceptible representations. We argue that this learning strategy is suboptimal in the realm of RS, since the required representations for different RS downstream tasks are often varied and complex. In this study, we proposed a unified SSL framework that is better suited for RS images representation learning. The proposed SSL framework, Contrastive Mask Image Distillation (CMID), is capable of learning representations with both global semantic separability and local spatial perceptibility by combining contrastive learning (CL) with masked image modeling (MIM) in a self-distillation way. Furthermore, our CMID learning framework is architecture-agnostic, which is compatible with both convolutional neural networks (CNN) and vision transformers (ViT), allowing CMID to be easily adapted to a variety of deep learning (DL) applications for RS understanding. Comprehensive experiments have been carried out on four downstream tasks (i.e. scene classification, semantic segmentation, object-detection, and change detection) and the results show that models pre-trained using CMID achieve better performance than other state-of-the-art SSL methods on multiple downstream tasks. The code and pre-trained models will be made available at https://github.com/NJU-LHRS/official-CMID to facilitate SSL research and speed up the development of RS images DL applications.

翻译：自监督学习（SSL）因其无需人工标注标签即可学习任务无关表征的能力，在遥感（RS）与地球观测（EO）领域受到广泛关注。然而，现有大多数RS SSL方法局限于学习全局语义可分离表征或局部空间可感知表征。我们认为，这种学习策略在RS领域并非最优，因为不同RS下游任务所需的表征往往多样且复杂。本研究提出了一种更适用于RS图像表征学习的统一SSL框架。该框架称为对比掩码图像蒸馏（CMID），通过将对比学习（CL）与掩码图像建模（MIM）以自蒸馏方式相结合，能够同时学习具有全局语义可分离性和局部空间可感知性的表征。此外，CMID学习框架具有架构无关性，兼容卷积神经网络（CNN）和视觉Transformer（ViT），使得CMID能够轻松适配各类用于RS理解的深度学习（DL）应用。我们在四个下游任务（即场景分类、语义分割、目标检测和变化检测）上开展了全面实验，结果表明，采用CMID预训练的模型在多个下游任务上均优于其他最先进的SSL方法。代码和预训练模型将在https://github.com/NJU-LHRS/official-CMID公开，以促进SSL研究并加速RS图像DL应用的发展。