Semantic segmentation is a computer vision task that associates a label with each pixel in an image. Modern approaches tend to introduce class embeddings into semantic segmentation for deeply utilizing category semantics, and regard supervised class masks as final predictions. In this paper, we explore the mechanism of class embeddings and have an insight that more explicit and meaningful class embeddings can be generated based on class masks purposely. Following this observation, we propose ECENet, a new segmentation paradigm, in which class embeddings are obtained and enhanced explicitly during interacting with multi-stage image features. Based on this, we revisit the traditional decoding process and explore inverted information flow between segmentation masks and class embeddings. Furthermore, to ensure the discriminability and informativity of features from backbone, we propose a Feature Reconstruction module, which combines intrinsic and diverse branches together to ensure the concurrence of diversity and redundancy in features. Experiments show that our ECENet outperforms its counterparts on the ADE20K dataset with much less computational cost and achieves new state-of-the-art results on PASCAL-Context dataset. The code will be released at https://gitee.com/mindspore/models and https://github.com/Carol-lyh/ECENet.

翻译：[translated abstract in Chinese] 语义分割是一项将标签与图像中每个像素相关联的计算机视觉任务。现代方法倾向于在语义分割中引入类别嵌入以深度利用类别语义，并将有监督的类别掩码作为最终预测结果。本文探究了类别嵌入的机制，并提出一种洞见：基于类别掩码可以有目的地生成更显式且有意义的类别嵌入。基于此观察，我们提出ECENet——一种新的分割范式，该方法在交互多阶段图像特征过程中显式地获取并增强类别嵌入。在此基础上，我们重新审视传统解码过程，探索分割掩码与类别嵌入之间的反向信息流。此外，为确保骨干网络提取特征的可判别性与信息量，我们提出特征重建模块，该模块将内在分支与多样性分支相结合，确保特征兼具多样性与冗余性。实验表明，我们的ECENet在ADE20K数据集上以更低的计算成本超越了同类方法，并在PASCAL-Context数据集上达到了最新最优结果。代码将在https://gitee.com/mindspore/models 和 https://github.com/Carol-lyh/ECENet 发布。