Existing knowledge distillation works for semantic segmentation mainly focus on transferring high-level contextual knowledge from teacher to student. However, low-level texture knowledge is also of vital importance for characterizing the local structural pattern and global statistical property, such as boundary, smoothness, regularity and color contrast, which may not be well addressed by high-level deep features. In this paper, we are intended to take full advantage of both structural and statistical texture knowledge and propose a novel Structural and Statistical Texture Knowledge Distillation (SSTKD) framework for semantic segmentation. Specifically, for structural texture knowledge, we introduce a Contourlet Decomposition Module (CDM) that decomposes low-level features with iterative Laplacian pyramid and directional filter bank to mine the structural texture knowledge. For statistical knowledge, we propose a Denoised Texture Intensity Equalization Module (DTIEM) to adaptively extract and enhance statistical texture knowledge through heuristics iterative quantization and denoised operation. Finally, each knowledge learning is supervised by an individual loss function, forcing the student network to mimic the teacher better from a broader perspective. Experiments show that the proposed method achieves state-of-the-art performance on Cityscapes, Pascal VOC 2012 and ADE20K datasets.

翻译：现有针对语义分割的知识蒸馏研究主要聚焦于从教师网络向学生网络迁移高层次上下文知识。然而，低层次纹理知识对于刻画局部结构模式和全局统计特性（如边界、平滑度、规律性及色彩对比度）至关重要，而这些特性可能无法通过高层深度特征充分表征。本文旨在充分利用结构性和统计性纹理知识，提出一种新颖的结构性与统计纹理知识蒸馏框架（SSTKD）用于语义分割。具体而言，在结构性纹理知识方面，我们引入Contourlet分解模块（CDM），通过迭代拉普拉斯金字塔和方向滤波器组对低层特征进行分解，以挖掘结构性纹理知识。在统计性知识方面，我们提出去噪纹理强度均衡模块（DTIEM），通过启发式迭代量化和去噪操作自适应提取并增强统计性纹理知识。最后，每种知识学习由独立损失函数监督，迫使学生网络从更广泛视角更好地模仿教师网络。实验表明，所提方法在Cityscapes、Pascal VOC 2012和ADE20K数据集上达到了最先进性能。