Diffusion models have attracted significant attention due to the remarkable ability to create content and generate data for tasks like image classification. However, the usage of diffusion models to generate the high-quality object detection data remains an underexplored area, where not only image-level perceptual quality but also geometric conditions such as bounding boxes and camera views are essential. Previous studies have utilized either copy-paste synthesis or layout-to-image (L2I) generation with specifically designed modules to encode the semantic layouts. In this paper, we propose the GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts and empower pre-trained text-to-image (T2I) diffusion models for high-quality detection data generation. Unlike previous L2I methods, our GeoDiffusion is able to encode not only the bounding boxes but also extra geometric conditions such as camera views in self-driving scenes. Extensive experiments demonstrate GeoDiffusion outperforms previous L2I methods while maintaining 4x training time faster. To the best of our knowledge, this is the first work to adopt diffusion models for layout-to-image generation with geometric conditions and demonstrate that L2I-generated images can be beneficial for improving the performance of object detectors.

翻译：扩散模型因其在图像分类等任务中出色的内容创建与数据生成能力而受到广泛关注。然而，利用扩散模型生成高质量的目标检测数据仍是一个未被充分探索的领域，其中不仅需要图像级别的感知质量，还需考虑边界框、相机视角等几何条件。以往研究要么采用复制-粘贴合成方法，要么使用专门设计的模块对语义布局进行编码的布局到图像（L2I）生成方法。本文提出GeoDiffusion，一个简单框架，能够将各种几何条件灵活转化为文本提示，并赋予预训练文本到图像（T2I）扩散模型生成高质量检测数据的能力。与以往的L2I方法不同，我们的GeoDiffusion不仅能编码边界框，还能编码自动驾驶场景中相机视角等额外几何条件。大量实验表明，GeoDiffusion在保持4倍训练速度提升的同时超越了以往的L2I方法。据我们所知，这是首个采用扩散模型进行带几何条件的布局到图像生成的研究，并证明了L2I生成的图像有助于提升目标检测器的性能。