Diffusion models have attracted significant attention due to the remarkable ability to create content and generate data for tasks like image classification. However, the usage of diffusion models to generate the high-quality object detection data remains an underexplored area, where not only image-level perceptual quality but also geometric conditions such as bounding boxes and camera views are essential. Previous studies have utilized either copy-paste synthesis or layout-to-image (L2I) generation with specifically designed modules to encode semantic layouts. In this paper, we propose GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts and empower pre-trained text-to-image (T2I) diffusion models for high-quality detection data generation. Unlike previous L2I methods, our GeoDiffusion is able to encode not only the bounding boxes but also extra geometric conditions such as camera views in self-driving scenes. Extensive experiments demonstrate GeoDiffusion outperforms previous L2I methods while maintaining 4x training time faster. To the best of our knowledge, this is the first work to adopt diffusion models for layout-to-image generation with geometric conditions and demonstrate that L2I-generated images can be beneficial for improving the performance of object detectors.

翻译：扩散模型因其在图像分类等任务中生成内容与数据的卓越能力而备受关注。然而，利用扩散模型生成高质量目标检测数据仍是一个有待探索的领域——这不仅需要图像级别的感知质量，还需满足边界框、相机视角等几何条件约束。现有研究或采用复制-粘贴合成方法，或通过定制化模块编码语义布局的布局到图像（L2I）生成技术。本文提出GeoDiffusion，一个简洁的框架，可灵活地将各类几何条件转化为文本提示，赋能预训练的文本到图像（T2I）扩散模型生成高质量检测数据。与先前L2I方法不同，本方案不仅能编码边界框，还能处理自动驾驶场景中相机视角等额外几何条件。大量实验表明，GeoDiffusion在保持4倍训练速度提升的同时，性能超越现有L2I方法。据我们所知，这是首个采用扩散模型实现几何条件约束下布局到图像生成的工作，并证实L2I生成图像可有效提升目标检测器的性能。