Few-shot object detection (FSOD) aims to expand an object detector for novel categories given only a few instances for training. The few training samples restrict the performance of FSOD model. Recent text-to-image generation models have shown promising results in generating high-quality images. How applicable these synthetic images are for FSOD tasks remains under-explored. This work extensively studies how synthetic images generated from state-of-the-art text-to-image generators benefit FSOD tasks. We focus on two perspectives: (1) How to use synthetic data for FSOD? (2) How to find representative samples from the large-scale synthetic dataset? We design a copy-paste-based pipeline for using synthetic data. Specifically, saliency object detection is applied to the original generated image, and the minimum enclosing box is used for cropping the main object based on the saliency map. After that, the cropped object is randomly pasted on the image, which comes from the base dataset. We also study the influence of the input text of text-to-image generator and the number of synthetic images used. To construct a representative synthetic training dataset, we maximize the diversity of the selected images via a sample-based and cluster-based method. However, the severe problem of high false positives (FP) ratio of novel categories in FSOD can not be solved by using synthetic data. We propose integrating CLIP, a zero-shot recognition model, into the FSOD pipeline, which can filter 90% of FP by defining a threshold for the similarity score between the detected object and the text of the predicted category. Extensive experiments on PASCAL VOC and MS COCO validate the effectiveness of our method, in which performance gain is up to 21.9% compared to the few-shot baseline.

翻译：小样本目标检测（FSOD）旨在仅通过少量实例训练，将目标检测器扩展到新类别。有限的训练样本制约了FSOD模型的性能。近年来，文本到图像生成模型在生成高质量图像方面展现出令人瞩目的效果，但这些合成图像对FSOD任务的适用性仍鲜有探究。本文系统研究了基于最先进文本到图像生成器产生的合成图像如何提升FSOD任务。我们聚焦于两个视角：（1）如何将合成数据用于FSOD？（2）如何从大规模合成数据集中选取代表性样本？我们设计了一种基于复制粘贴的合成数据使用流程。具体而言，对原始生成图像应用显著性目标检测，并基于显著性图使用最小外接矩形裁剪主要目标。随后，将裁剪出的目标随机粘贴到源自基础数据集的图像上。我们还研究了文本到图像生成器的输入文本以及合成图像使用数量带来的影响。为构建具有代表性的合成训练数据集，我们通过基于样本和基于聚类的方法最大化所选图像的多样性。然而，合成数据无法解决FSOD中新类别高虚警率（FP）的严峻问题。我们提出将零样本识别模型CLIP集成到FSOD流程中，通过设定检测目标与预测类别文本之间相似度分数的阈值，可过滤90%的虚警。在PASCAL VOC和MS COCO上的大量实验验证了方法的有效性，相较小样本基线，性能提升高达21.9%。