Semantic segmentation has witnessed tremendous progress due to the proposal of various advanced network architectures. However, they are extremely hungry for delicate annotations to train, and the acquisition is laborious and unaffordable. Therefore, we present FreeMask in this work, which resorts to synthetic images from generative models to ease the burden of both data collection and annotation procedures. Concretely, we first synthesize abundant training images conditioned on the semantic masks provided by realistic datasets. This yields extra well-aligned image-mask training pairs for semantic segmentation models. We surprisingly observe that, solely trained with synthetic images, we already achieve comparable performance with real ones (e.g., 48.3 vs. 48.5 mIoU on ADE20K, and 49.3 vs. 50.5 on COCO-Stuff). Then, we investigate the role of synthetic images by joint training with real images, or pre-training for real images. Meantime, we design a robust filtering principle to suppress incorrectly synthesized regions. In addition, we propose to inequally treat different semantic masks to prioritize those harder ones and sample more corresponding synthetic images for them. As a result, either jointly trained or pre-trained with our filtered and re-sampled synthesized images, segmentation models can be greatly enhanced, e.g., from 48.7 to 52.0 on ADE20K. Code is available at https://github.com/LiheYoung/FreeMask.

翻译：语义分割因各类先进网络架构的提出而取得了巨大进展，但此类模型对精细标注数据极度渴求，而获取此类数据的标注过程既费力又成本高昂。为此，本文提出FreeMask方法，借助生成模型的合成图像来减轻数据收集和标注流程的负担。具体而言，我们首先基于真实数据集提供的语义掩码，合成大量用于训练的图像。这为语义分割模型提供了额外且完美配对的图像-掩码训练样本。令人惊讶的是，仅使用合成图像训练，我们已能达到与真实图像相当的性能（例如，在ADE20K上为48.3对48.5 mIoU，在COCO-Stuff上为49.3对50.5）。随后，我们通过联合训练（合成图像与真实图像共同训练）或预训练（以合成图像预训练后微调于真实图像）探究了合成图像的作用。同时，我们设计了一种鲁棒的过滤原则以抑制错误合成的区域。此外，我们提出对不同语义掩码进行非等价处理，优先关注较难掩码并为其采样更多对应的合成图像。最终，无论是采用过滤与重采样后的合成图像进行联合训练还是预训练，分割模型均得到显著增强，例如在ADE20K上性能从48.7提升至52.0。代码发布于https://github.com/LiheYoung/FreeMask。