This paper proposes a new knowledge distillation method tailored for image semantic segmentation, termed Intra- and Inter-Class Knowledge Distillation (I2CKD). The focus of this method is on capturing and transferring knowledge between the intermediate layers of teacher (cumbersome model) and student (compact model). For knowledge extraction, we exploit class prototypes derived from feature maps. To facilitate knowledge transfer, we employ a triplet loss in order to minimize intra-class variances and maximize inter-class variances between teacher and student prototypes. Consequently, I2CKD enables the student to better mimic the feature representation of the teacher for each class, thereby enhancing the segmentation performance of the compact network. Extensive experiments on three segmentation datasets, i.e., Cityscapes, Pascal VOC and CamVid, using various teacher-student network pairs demonstrate the effectiveness of the proposed method.

翻译：本文提出一种专用于图像语义分割的新型知识蒸馏方法，称为类内与类间知识蒸馏（Intra- and Inter-Class Knowledge Distillation, I2CKD）。该方法的核心在于捕捉并传递教师网络（复杂模型）与学生网络（紧凑模型）中间层之间的知识。为提取知识，我们利用从特征图中导出的类原型（class prototypes）。为促进知识迁移，我们采用三元组损失（triplet loss）来最小化教师与学生原型之间的类内方差，同时最大化其类间方差。由此，I2CKD使学生网络能够更准确地模仿教师网络对每个类别的特征表征，从而提升紧凑网络的分割性能。在三个分割数据集（Cityscapes、Pascal VOC和CamVid）上，使用多种教师-学生网络对进行的大量实验验证了所提方法的有效性。