Instance segmentation is data-hungry, and as model capacity increases, data scale becomes crucial for improving the accuracy. Most instance segmentation datasets today require costly manual annotation, limiting their data scale. Models trained on such data are prone to overfitting on the training set, especially for those rare categories. While recent works have delved into exploiting generative models to create synthetic datasets for data augmentation, these approaches do not efficiently harness the full potential of generative models. To address these issues, we introduce a more efficient strategy to construct generative datasets for data augmentation, termed DiverGen. Firstly, we provide an explanation of the role of generative data from the perspective of distribution discrepancy. We investigate the impact of different data on the distribution learned by the model. We argue that generative data can expand the data distribution that the model can learn, thus mitigating overfitting. Additionally, we find that the diversity of generative data is crucial for improving model performance and enhance it through various strategies, including category diversity, prompt diversity, and generative model diversity. With these strategies, we can scale the data to millions while maintaining the trend of model performance improvement. On the LVIS dataset, DiverGen significantly outperforms the strong model X-Paste, achieving +1.1 box AP and +1.1 mask AP across all categories, and +1.9 box AP and +2.5 mask AP for rare categories.

翻译：实例分割任务对数据需求量大，随着模型容量的提升，数据规模对提高精度至关重要。当前多数实例分割数据集依赖昂贵的人工标注，限制了数据规模。在此类数据上训练的模型容易过拟合训练集，尤其是针对稀有类别。尽管近期工作已探索利用生成模型构建合成数据集进行数据增强，但这些方法未能高效发挥生成模型的全部潜力。为此，我们提出一种更高效的策略来构建用于数据增强的生成数据集，称为DiverGen。首先，我们从分布差异的角度解释生成数据的作用机制，探究不同数据对模型所学习分布的影响，论证生成数据可扩展模型可学习的数据分布范围，从而缓解过拟合。此外，我们发现生成数据的多样性对提升模型性能至关重要，并通过多种策略增强其多样性，包括类别多样性、提示多样性和生成模型多样性。利用这些策略，我们可将数据规模扩展至百万级，同时保持模型性能提升趋势。在LVIS数据集上，DiverGen显著超越强基线模型X-Paste，全类别箱体AP和掩膜AP分别提升+1.1和+1.1，稀有类别箱体AP和掩膜AP分别提升+1.9和+2.5。