In recent years, self-supervision has drawn a lot of attention in remote sensing society due to its ability to reduce the demand of exact labels in supervised deep learning model training. Self-supervision methods generally utilize image-level information to pretrain models in an unsupervised fashion. Though these pretrained encoders show effectiveness in many downstream tasks, their performance on segmentation tasks is often not as good as that on classification tasks. On the other hand, many easily available label sources (e.g., automatic labeling tools and land cover land use products) exist, which can provide a large amount of noisy labels for segmentation model training. In this work, we propose to explore the under-exploited potential of noisy labels for segmentation task specific pretraining, and exam its robustness when confronted with mismatched categories and different decoders during fine-tuning. Specifically, we inspect the impacts of noisy labels on different layers in supervised model training to serve as the basis of our work. Experiments on two datasets indicate the effectiveness of task specific supervised pretraining with noisy labels. The findings are expected to shed light on new avenues for improving the accuracy and versatility of pretraining strategies for remote sensing image segmentation.

翻译：近年来，自监督学习因其能够减少监督深度学习模型训练中对精确标签的需求，在遥感领域引起了广泛关注。自监督方法通常利用图像级信息以无监督方式预训练模型。尽管这些预训练编码器在许多下游任务中表现出有效性，但其在分割任务上的表现往往不如分类任务理想。另一方面，存在大量易于获取的标签来源（如自动标注工具和土地覆盖/土地利用产品），这些来源可为分割模型训练提供大量噪声标签。本研究旨在探索噪声标签在分割任务特定预训练中未被充分利用的潜力，并检验其在微调过程中面对类别不匹配及不同解码器时的鲁棒性。具体而言，我们通过分析噪声标签对监督模型训练中不同层次的影响，为研究奠定基础。两个数据集上的实验表明，利用噪声标签进行任务特定监督预训练具有有效性。这些发现有望为改善遥感图像分割预训练策略的准确性与通用性开辟新途径。