Semi-supervised semantic segmentation methods leverage unlabeled data by pseudo-labeling them. Thus the success of these methods hinges on the reliablility of the pseudo-labels. Existing methods mostly choose high-confidence pixels in an effort to avoid erroneous pseudo-labels. However, high confidence does not guarantee correct pseudo-labels especially in the initial training iterations. In this paper, we propose a novel approach to reliably learn from pseudo-labels. First, we unify the predictions from a trained object detector and a semantic segmentation model to identify reliable pseudo-label pixels. Second, we assign different learning weights to pseudo-labeled pixels to avoid noisy training signals. To determine these weights, we first use the reliable pseudo-label pixels identified from the first step and labeled pixels to construct a prototype for each class. Then, the per-pixel weight is the structural similarity between the pixel and the prototype measured via rank-statistics similarity. This metric is robust to noise, making it better suited for comparing features from unlabeled images, particularly in the initial training phases where wrong pseudo labels are prone to occur. We show that our method can be easily integrated into four semi-supervised semantic segmentation frameworks, and improves them in both Cityscapes and Pascal VOC datasets.

翻译：半监督语义分割方法通过伪标注利用未标注数据，因此这些方法的成功取决于伪标签的可靠性。现有方法大多选择高置信度像素以避免错误伪标签，然而高置信度并不能保证伪标签的正确性，尤其在初始训练阶段。本文提出一种从伪标签中可靠学习的新方法。首先，我们整合训练好的目标检测器与语义分割模型的预测结果以识别可靠的伪标签像素。其次，我们为伪标注像素分配不同的学习权重以避免噪声训练信号。为确定这些权重，我们首先利用第一步识别的可靠伪标签像素及已标注像素构建每个类别的原型。随后，通过秩统计相似度度量像素与原型之间的结构相似性作为逐像素权重。该指标对噪声具有鲁棒性，使其更适合比较未标注图像的特征，特别是在易出现错误伪标签的初始训练阶段。实验表明，我们的方法能够轻松集成到四种半监督语义分割框架中，并在Cityscapes和Pascal VOC数据集上均提升了其性能。