We present Generative Semantic Segmentation (GSS), a generative learning approach for semantic segmentation. Uniquely, we cast semantic segmentation as an image-conditioned mask generation problem. This is achieved by replacing the conventional per-pixel discriminative learning with a latent prior learning process. Specifically, we model the variational posterior distribution of latent variables given the segmentation mask. To that end, the segmentation mask is expressed with a special type of image (dubbed as maskige). This posterior distribution allows to generate segmentation masks unconditionally. To achieve semantic segmentation on a given image, we further introduce a conditioning network. It is optimized by minimizing the divergence between the posterior distribution of maskige (i.e., segmentation masks) and the latent prior distribution of input training images. Extensive experiments on standard benchmarks show that our GSS can perform competitively to prior art alternatives in the standard semantic segmentation setting, whilst achieving a new state of the art in the more challenging cross-domain setting.

翻译：我们提出生成式语义分割（Generative Semantic Segmentation, GSS），这是一种用于语义分割的生成式学习方法。独特之处在于，我们将语义分割视为一种图像条件的掩码生成问题。这是通过用潜在先验学习过程替代传统的逐像素判别学习来实现的。具体而言，我们建模给定分割掩码的潜在变量的变分后验分布。为此，分割掩码用一种特殊类型的图像（称为掩码图）表示。该后验分布允许无条件地生成分割掩码。为了在给定图像上实现语义分割，我们进一步引入了一个条件网络。它通过最小化掩码图（即分割掩码）的后验分布与输入训练图像的潜在先验分布之间的散度进行优化。在标准基准上的大量实验表明，我们的GSS在标准语义分割设置中能够与现有先进方法竞争，同时在更具挑战性的跨域设置中达到了新的最佳水平。