We propose SegGen, a highly-effective training data generation method for image segmentation, which pushes the performance limits of state-of-the-art segmentation models to a significant extent. SegGen designs and integrates two data generation strategies: MaskSyn and ImgSyn. (i) MaskSyn synthesizes new mask-image pairs via our proposed text-to-mask generation model and mask-to-image generation model, greatly improving the diversity in segmentation masks for model supervision; (ii) ImgSyn synthesizes new images based on existing masks using the mask-to-image generation model, strongly improving image diversity for model inputs. On the highly competitive ADE20K and COCO benchmarks, our data generation method markedly improves the performance of state-of-the-art segmentation models in semantic segmentation, panoptic segmentation, and instance segmentation. Notably, in terms of the ADE20K mIoU, Mask2Former R50 is largely boosted from 47.2 to 49.9 (+2.7); Mask2Former Swin-L is also significantly increased from 56.1 to 57.4 (+1.3). These promising results strongly suggest the effectiveness of our SegGen even when abundant human-annotated training data is utilized. Moreover, training with our synthetic data makes the segmentation models more robust towards unseen domains. Project website: https://seggenerator.github.io

翻译：我们提出SegGen，一种面向图像分割的高效训练数据生成方法，该方法显著推动了当前最优分割模型的性能上限。SegGen设计并整合了两种数据生成策略：MaskSyn与ImgSyn。（i）MaskSyn通过我们提出的文本到掩码生成模型与掩码到图像生成模型合成新的掩码-图像对，极大提升了分割掩码的多样性以用于模型监督；（ii）ImgSyn基于现有掩码利用掩码到图像生成模型合成新图像，显著增强了模型输入的图像多样性。在极具竞争力的ADE20K与COCO基准测试中，我们的数据生成方法在语义分割、全景分割与实例分割任务上均显著提升了最优分割模型的性能。值得注意的是，在ADE20K mIoU指标上，Mask2Former R50从47.2大幅提升至49.9（+2.7）；Mask2Former Swin-L亦从56.1显著提升至57.4（+1.3）。即使在使用大量人工标注训练数据的情况下，这些优异结果仍有力证明了SegGen的有效性。此外，使用合成数据进行训练使得分割模型对未见过域具有更强的鲁棒性。项目网站：https://seggenerator.github.io