Few-shot object detection (FSOD) aims to expand an object detector for novel categories given only a few instances for training. The few training samples restrict the performance of FSOD model. Recent text-to-image generation models have shown promising results in generating high-quality images. How applicable these synthetic images are for FSOD tasks remains under-explored. This work extensively studies how synthetic images generated from state-of-the-art text-to-image generators benefit FSOD tasks. We focus on two perspectives: (1) How to use synthetic data for FSOD? (2) How to find representative samples from the large-scale synthetic dataset? We design a copy-paste-based pipeline for using synthetic data. Specifically, saliency object detection is applied to the original generated image, and the minimum enclosing box is used for cropping the main object based on the saliency map. After that, the cropped object is randomly pasted on the image, which comes from the base dataset. We also study the influence of the input text of text-to-image generator and the number of synthetic images used. To construct a representative synthetic training dataset, we maximize the diversity of the selected images via a sample-based and cluster-based method. However, the severe problem of high false positives (FP) ratio of novel categories in FSOD can not be solved by using synthetic data. We propose integrating CLIP, a zero-shot recognition model, into the FSOD pipeline, which can filter 90% of FP by defining a threshold for the similarity score between the detected object and the text of the predicted category. Extensive experiments on PASCAL VOC and MS COCO validate the effectiveness of our method, in which performance gain is up to 21.9% compared to the few-shot baseline.

翻译：少样本目标检测（FSOD）旨在仅通过少量训练实例扩展目标检测器以识别新类别。有限的训练样本制约了FSOD模型的性能。近年来的文本到图像生成模型在生成高质量图像方面展现出显著效果，但合成图像对FSOD任务的适用性仍有待探究。本文系统研究了基于最先进的文本到图像生成模型产生的合成图像如何提升FSOD任务性能。我们聚焦两个核心问题：（1）如何将合成数据应用于FSOD？（2）如何从大规模合成数据集中筛选代表性样本？我们设计了一种基于复制-粘贴的合成数据流水线：首先对原始生成图像进行显著性目标检测，基于显著性图采用最小外接矩形裁剪主要目标；随后将裁剪目标随机粘贴至基础数据集的图像中。我们还分析了文本到图像生成器的输入文本及合成图像数量对性能的影响。为构建具有代表性的合成训练数据集，我们通过基于样本和基于聚类的方法最大化所选图像的多样性。然而，合成数据无法解决FSOD中新类别高误检率（FP）的严重问题。我们提出将零样本识别模型CLIP集成至FSOD流水线中，通过定义检测目标与预测类别文本的相似度阈值，可过滤90%的误检。在PASCAL VOC和MS COCO数据集上的大量实验验证了该方法的有效性，相较于少样本基线模型，性能提升最高达21.9%。