We present a method for expanding a dataset by incorporating knowledge from the wide distribution of pre-trained latent diffusion models. Data augmentations typically incorporate inductive biases about the image formation process into the training (e.g. translation, scaling, colour changes, etc.). Here, we go beyond simple pixel transformations and introduce the concept of instance-level data augmentation by repainting parts of the image at the level of object instances. The method combines a conditional diffusion model with depth and edge maps control conditioning to seamlessly repaint individual objects inside the scene, being applicable to any segmentation or detection dataset. Used as a data augmentation method, it improves the performance and generalization of the state-of-the-art salient object detection, semantic segmentation and object detection models. By redrawing all privacy-sensitive instances (people, license plates, etc.), the method is also applicable for data anonymization. We also release fully synthetic and anonymized expansions for popular datasets: COCO, Pascal VOC and DUTS.

翻译：我们提出了一种通过整合预训练潜在扩散模型广泛分布知识来扩展数据集的方法。传统数据增强通常将图像形成过程的归纳偏置引入训练（例如平移、缩放、色彩变换等）。本文超越了简单的像素变换，通过在对象实例层面重绘图像局部区域，提出了实例级数据增强的概念。该方法结合条件扩散模型与深度及边缘图控制条件，实现对场景中独立对象的无缝重绘，可适用于任何分割或检测数据集。作为数据增强方法，它能提升当前最先进的显著目标检测、语义分割和目标检测模型的性能与泛化能力。通过重绘所有隐私敏感实例（人物、车牌等），该方法还可应用于数据匿名化处理。我们同时发布了针对流行数据集（COCO、Pascal VOC和DUTS）的完全合成与匿名化扩展版本。