The evolution of semantic segmentation has long been dominated by learning more discriminative image representations for classifying each pixel. Despite the prominent advancements, the priors of segmentation masks themselves, e.g., geometric and semantic constraints, are still under-explored. In this paper, we propose to ameliorate the semantic segmentation quality of existing discriminative approaches with a mask prior modeled by a recently-developed denoising diffusion generative model. Beginning with a unified architecture that adapts diffusion models for mask prior modeling, we focus this work on a specific instantiation with discrete diffusion and identify a variety of key design choices for its successful application. Our exploratory analysis revealed several important findings, including: (1) a simple integration of diffusion models into semantic segmentation is not sufficient, and a poorly-designed diffusion process might lead to degradation in segmentation performance; (2) during the training, the object to which noise is added is more important than the type of noise; (3) during the inference, the strict diffusion denoising scheme may not be essential and can be relaxed to a simpler scheme that even works better. We evaluate the proposed prior modeling with several off-the-shelf segmentors, and our experimental results on ADE20K and Cityscapes demonstrate that our approach could achieve competitively quantitative performance and more appealing visual quality.

翻译：语义分割的发展长期以来主导于学习更具判别性的图像表示以对每个像素进行分类。尽管取得了显著进展，但分割掩码本身的先验信息（例如几何约束和语义约束）仍未得到充分探索。本文提出利用最近发展的去噪扩散生成模型所建模的掩码先验，来提升现有判别式方法的语义分割质量。我们从统一的适配扩散模型进行掩码先验建模的架构出发，专注于离散扩散的具体实例化，并为其成功应用识别了一系列关键设计选择。探索性分析揭示了若干重要发现，包括：（1）简单地将扩散模型集成到语义分割中并不足够，设计不当的扩散过程可能导致分割性能下降；（2）训练过程中，添加噪声的对象比噪声类型更为重要；（3）推理过程中，严格的扩散去噪方案可能并非必要，可简化为效果更优的简单方案。我们利用多个现成分割器对所提出的先验建模进行了评估，在ADE20K和Cityscapes上的实验结果表明，我们的方法能够取得具有竞争力的量化性能以及更吸引人的视觉质量。