Diffusion models have attracted significant attention due to their remarkable ability to create content and generate data for tasks such as image classification. However, the usage of diffusion models to generate high-quality object detection data remains an underexplored area, where not only the image-level perceptual quality but also geometric conditions such as bounding boxes and camera views are essential. Previous studies have utilized either copy-paste synthesis or layout-to-image (L2I) generation with specifically designed modules to encode semantic layouts. In this paper, we propose GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts and empower the pre-trained text-to-image (T2I) diffusion models for high-quality detection data generation. Unlike previous L2I methods, our GeoDiffusion is able to encode not only bounding boxes but also extra geometric conditions such as camera views in self-driving scenes. Extensive experiments demonstrate GeoDiffusion outperforms previous L2I methods while maintaining 4x training time faster. To the best of our knowledge, this is the first work to adopt diffusion models for layout-to-image generation with geometric conditions and demonstrate that L2I-generated images can be beneficial for improving the performance of object detectors.

翻译：扩散模型因其在图像分类等任务中生成内容和数据的卓越能力而受到广泛关注。然而，利用扩散模型生成高质量的目标检测数据仍是一个尚未充分探索的领域，其中不仅需要图像级别的感知质量，还需要边界框和相机视角等几何条件。先前的研究要么采用复制粘贴合成，要么使用专门设计的模块进行布局到图像（L2I）生成以编码语义布局。在本文中，我们提出GeoDiffusion，一个简单框架，能够灵活地将各种几何条件转化为文本提示，并赋能预训练的文本到图像（T2I）扩散模型用于高质量检测数据生成。与先前的L2I方法不同，我们的GeoDiffusion不仅能够编码边界框，还能编码自动驾驶场景中的相机视角等额外几何条件。大量实验表明，GeoDiffusion在保持4倍训练速度提升的同时，优于先前的L2I方法。据我们所知，这是首个采用扩散模型实现带几何条件的布局到图像生成的工作，并证明了L2I生成图像有助于提升目标检测器的性能。