We consider unsupervised domain adaptation (UDA) for semantic segmentation in which the model is trained on a labeled source dataset and adapted to an unlabeled target dataset. Unfortunately, current self-training methods are susceptible to misclassified pseudo-labels resulting from erroneous predictions. Since certain classes are typically associated with less reliable predictions in UDA, reducing the impact of such pseudo-labels without skewing the training towards some classes is notoriously difficult. To this end, we propose an extensive cut-and-paste strategy (ECAP) to leverage reliable pseudo-labels through data augmentation. Specifically, ECAP maintains a memory bank of pseudo-labeled target samples throughout training and cut-and-pastes the most confident ones onto the current training batch. We implement ECAP on top of the recent method MIC and boost its performance on two synthetic-to-real domain adaptation benchmarks. Notably, MIC+ECAP reaches an unprecedented performance of 69.1 mIoU on the Synthia->Cityscapes benchmark. Our code is available at https://github.com/ErikBrorsson/ECAP.
翻译:我们考虑无监督域自适应在语义分割中的应用,即利用有标签源域数据集训练模型,并使其适应无标签目标域数据集。然而,当前的自训练方法易受错误预测导致的误分类伪标签影响。由于在无监督域自适应中,某些类别通常与可靠性较低的预测相关联,因此在不导致训练偏向特定类别的前提下减少此类伪标签的影响极为困难。为此,我们提出一种广泛切割粘贴策略(ECAP),通过数据增强有效利用可靠伪标签。具体而言,ECAP在训练过程中维护一个伪标签目标样本记忆库,并将其中置信度最高的样本切割粘贴至当前训练批次中。我们在最新方法MIC基础上实现ECAP,并在两个合成到真实域自适应基准任务上提升了其性能。值得注意的是,MIC+ECAP在Synthia→Cityscapes基准上达到了69.1 mIoU的突破性表现。我们的代码已开源至https://github.com/ErikBrorsson/ECAP。