We present Generative Semantic Segmentation (GSS), a generative learning approach for semantic segmentation. Uniquely, we cast semantic segmentation as an image-conditioned mask generation problem. This is achieved by replacing the conventional per-pixel discriminative learning with a latent prior learning process. Specifically, we model the variational posterior distribution of latent variables given the segmentation mask. To that end, the segmentation mask is expressed with a special type of image (dubbed as maskige). This posterior distribution allows to generate segmentation masks unconditionally. To achieve semantic segmentation on a given image, we further introduce a conditioning network. It is optimized by minimizing the divergence between the posterior distribution of maskige (i.e., segmentation masks) and the latent prior distribution of input training images. Extensive experiments on standard benchmarks show that our GSS can perform competitively to prior art alternatives in the standard semantic segmentation setting, whilst achieving a new state of the art in the more challenging cross-domain setting.

翻译：我们提出生成式语义分割（GSS），一种用于语义分割的生成式学习方法。其独特性在于，我们将语义分割重新定义为一种图像条件化的掩膜生成问题。这一目标通过用潜在先验学习过程替代传统的逐像素判别学习来实现。具体而言，我们对给定分割掩膜条件下潜变量的变分后验分布进行建模。为此，分割掩膜被表达为一种特殊类型的图像（称为掩膜图像）。该后验分布允许无条件地生成分割掩膜。为在给定图像上实现语义分割，我们进一步引入了一个条件网络。该网络通过最小化掩膜图像（即分割掩膜）的后验分布与输入训练图像的潜在先验分布之间的散度进行优化。在标准基准上的大量实验表明，我们的GSS在标准语义分割设置中可与现有替代方法竞争，同时在更具挑战性的跨领域设置中达到了最新最优水平。