The popular methods for semi-supervised semantic segmentation mostly adopt a unitary network model using convolutional neural networks (CNNs) and enforce consistency of the model's predictions over perturbations applied to the inputs or model. However, such a learning paradigm suffers from two critical limitations: a) learning the discriminative features for the unlabeled data; b) learning both global and local information from the whole image. In this paper, we propose a novel Semi-supervised Learning (SSL) approach, called Transformer-CNN Cohort (TCC), that consists of two students with one based on the vision transformer (ViT) and the other based on the CNN. Our method subtly incorporates the multi-level consistency regularization on the predictions and the heterogeneous feature spaces via pseudo-labeling for the unlabeled data. First, as the inputs of the ViT student are image patches, the feature maps extracted encode crucial class-wise statistics. To this end, we propose class-aware feature consistency distillation (CFCD) that first leverages the outputs of each student as the pseudo labels and generates class-aware feature (CF) maps for knowledge transfer between the two students. Second, as the ViT student has more uniform representations for all layers, we propose consistency-aware cross distillation (CCD) to transfer knowledge between the pixel-wise predictions from the cohort. We validate the TCC framework on Cityscapes and Pascal VOC 2012 datasets, which outperforms existing SSL methods by a large margin.

翻译：半监督语义分割的主流方法大多采用基于卷积神经网络（CNN）的单一网络模型，并对输入或模型施加扰动后的一致性预测进行约束。然而，这种学习范式存在两个关键限制：a）难以从未标记数据中学习判别性特征；b）难以从整幅图像中同时学习全局与局部信息。本文提出一种新颖的半监督学习方法——Transformer-CNN 队列（TCC），该方法包含两个学生模型：一个基于视觉Transformer（ViT），另一个基于CNN。该方法通过伪标签技术，巧妙地将多层级一致性正则化作用于预测结果与异构特征空间。首先，由于ViT学生模型的输入为图像块，其提取的特征图编码了关键的类别统计信息。为此我们提出类别感知特征一致性蒸馏（CFCD），该方法首先利用每个学生模型的输出作为伪标签，生成类别感知特征（CF）图，以实现两个学生模型间的知识迁移。其次，针对ViT学生模型各层具有更均匀表征的特点，我们提出一致性感知交叉蒸馏（CCD），在队列的像素级预测之间进行知识迁移。我们在Cityscapes和Pascal VOC 2012数据集上验证了TCC框架，其性能大幅超越现有半监督学习方法。