Diffusion models have attracted significant attention due to their remarkable ability to create content and generate data for tasks such as image classification. However, the usage of diffusion models to generate high-quality object detection data remains an underexplored area, where not only the image-level perceptual quality but also geometric conditions such as bounding boxes and camera views are essential. Previous studies have utilized either copy-paste synthesis or layout-to-image (L2I) generation with specifically designed modules to encode semantic layouts. In this paper, we propose GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts and empower the pre-trained text-to-image (T2I) diffusion models for high-quality detection data generation. Unlike previous L2I methods, our GeoDiffusion is able to encode not only bounding boxes but also extra geometric conditions such as camera views in self-driving scenes. Extensive experiments demonstrate GeoDiffusion outperforms previous L2I methods while maintaining 4x training time faster. To the best of our knowledge, this is the first work to adopt diffusion models for layout-to-image generation with geometric conditions and demonstrate that L2I-generated images can be beneficial for improving the performance of object detectors.

翻译：扩散模型因其在内容创建和图像分类等任务数据生成方面的显著能力而备受关注。然而，利用扩散模型生成高质量的目标检测数据仍是一个尚未充分探索的领域，其中不仅需要图像层面的感知质量，还需要边界框和相机视角等几何条件。以往的研究要么采用复制-粘贴合成，要么使用专门设计的模块进行布局到图像（L2I）生成以编码语义布局。在本文中，我们提出了GeoDiffusion，一个简单的框架，能够灵活地将各种几何条件转换为文本提示，并赋能预训练的文本到图像（T2I）扩散模型生成高质量检测数据。与以往的L2I方法不同，我们的GeoDiffusion不仅能够编码边界框，还能编码自动驾驶场景中的相机视角等额外几何条件。大量实验表明，GeoDiffusion在保持训练速度提升4倍的同时，性能优于以往的L2I方法。据我们所知，这是首个采用扩散模型在几何条件下进行布局到图像生成的工作，并证明了L2I生成的图像有助于提升目标检测器的性能。