One-to-one matching is a crucial design in DETR-like object detection frameworks. It enables the DETR to perform end-to-end detection. However, it also faces challenges of lacking positive sample supervision and slow convergence speed. Several recent works proposed the one-to-many matching mechanism to accelerate training and boost detection performance. We revisit these methods and model them in a unified format of augmenting the object queries. In this paper, we propose two methods that realize one-to-many matching from a different perspective of augmenting images or image features. The first method is One-to-many Matching via Data Augmentation (denoted as DataAug-DETR). It spatially transforms the images and includes multiple augmented versions of each image in the same training batch. Such a simple augmentation strategy already achieves one-to-many matching and surprisingly improves DETR's performance. The second method is One-to-many matching via Feature Augmentation (denoted as FeatAug-DETR). Unlike DataAug-DETR, it augments the image features instead of the original images and includes multiple augmented features in the same batch to realize one-to-many matching. FeatAug-DETR significantly accelerates DETR training and boosts detection performance while keeping the inference speed unchanged. We conduct extensive experiments to evaluate the effectiveness of the proposed approach on DETR variants, including DAB-DETR, Deformable-DETR, and H-Deformable-DETR. Without extra training data, FeatAug-DETR shortens the training convergence periods of Deformable-DETR to 24 epochs and achieves 58.3 AP on COCO val2017 set with Swin-L as the backbone.
翻译:一对多匹配是DETR类目标检测框架中的关键设计,它使DETR能够实现端到端检测。然而,该方法也面临缺乏正样本监督和收敛速度慢的挑战。近期多项研究提出一对多匹配机制以加速训练并提升检测性能。本文重新审视这些方法,并将其建模为统一的查询增强范式。在此背景下,我们提出两种从图像或图像特征增强不同角度实现一对多匹配的方法。第一种方法为"数据增强式一对多匹配"(记为DataAug-DETR),通过对图像进行空间变换,将每个图像的多个增强版本纳入同一训练批次。这种简单的增强策略即可实现一对多匹配,并显著提升DETR性能。第二种方法为"特征增强式一对多匹配"(记为FeatAug-DETR),与DataAug-DETR不同,该方法增强的是图像特征而非原始图像,通过在同一批次中包含多个增强特征来实现一对多匹配。FeatAug-DETR在保持推理速度不变的前提下,显著加速了DETR训练并提升了检测性能。我们针对DETR变体(包括DAB-DETR、Deformable-DETR和H-Deformable-DETR)开展了大量实验以评估所提方法的有效性。在无额外训练数据的情况下,FeatAug-DETR将Deformable-DETR的训练收敛周期缩短至24个epoch,并在以Swin-L为骨干网络的COCO val2017数据集上达到58.3 AP。