Diffusion models have attracted significant attention due to their remarkable ability to create content and generate data for tasks such as image classification. However, the usage of diffusion models to generate high-quality object detection data remains an underexplored area, where not only the image-level perceptual quality but also geometric conditions such as bounding boxes and camera views are essential. Previous studies have utilized either copy-paste synthesis or layout-to-image (L2I) generation with specifically designed modules to encode semantic layouts. In this paper, we propose GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts and empower the pre-trained text-to-image (T2I) diffusion models for high-quality detection data generation. Unlike previous L2I methods, our GeoDiffusion is able to encode not only bounding boxes but also extra geometric conditions such as camera views in self-driving scenes. Extensive experiments demonstrate GeoDiffusion outperforms previous L2I methods while maintaining 4x training time faster. To the best of our knowledge, this is the first work to adopt diffusion models for layout-to-image generation with geometric conditions and demonstrate that L2I-generated images can be beneficial for improving the performance of object detectors.

翻译：扩散模型因其在图像分类等任务中生成内容与数据的卓越能力而备受关注。然而，利用扩散模型生成高质量的目标检测数据仍是一个未充分探索的领域，在此类任务中，不仅需要图像级别的感知质量，还要求边界框、相机视角等几何条件。以往的研究采用复制粘贴合成或基于特定编码语义布局的布局到图像（L2I）生成方法。本文提出GeoDiffusion这一简洁框架，可灵活地将多种几何条件转换为文本提示，并赋予预训练文本到图像（T2I）扩散模型生成高质量检测数据的能力。与以往L2I方法不同，GeoDiffusion不仅能编码边界框，还能编码自动驾驶场景中的相机视角等额外几何条件。大量实验表明，GeoDiffusion在保持4倍训练速度提升的同时，性能优于以往的L2I方法。据我们所知，这是首个将扩散模型应用于带几何条件的布局到图像生成，并证明L2I生成图像可有效改善目标检测器性能的研究。