In this paper, we aim to study how to build a strong instance segmenter with minimal training time and GPUs, as opposed to the majority of current approaches that pursue more accurate instance segmenter by building more advanced frameworks at the cost of longer training time and higher GPU requirements. To achieve this, we introduce a simple and general framework, termed Mask Frozen-DETR, which can convert any existing DETR-based object detection model into a powerful instance segmentation model. Our method only requires training an additional lightweight mask network that predicts instance masks within the bounding boxes given by a frozen DETR-based object detector. Remarkably, our method outperforms the state-of-the-art instance segmentation method Mask DINO in terms of performance on the COCO test-dev split (55.3% vs. 54.7%) while being over 10X times faster to train. Furthermore, all of our experiments can be trained using only one Tesla V100 GPU with 16 GB of memory, demonstrating the significant efficiency of our proposed framework.

翻译：本文旨在研究如何以最短的训练时间和最少的GPU资源构建强大的实例分割器，这与当前大多数方法形成鲜明对比——后者通过构建更先进的框架来追求更高的实例分割精度，却以更长的训练时间和更高的GPU需求为代价。为此，我们提出一个简单且通用的框架，称为掩码冻结-DETR（Mask Frozen-DETR），它能将任何现有基于DETR的目标检测模型转化为强大的实例分割模型。我们的方法仅需训练一个额外的轻量级掩码网络，用于在冻结的基于DETR的目标检测器所给出的边界框内预测实例掩码。值得注意的是，我们的方法在COCO测试集分割集上性能优于当前最先进的实例分割方法Mask DINO（55.3%对比54.7%），同时训练速度提升超过10倍。此外，我们所有实验仅需使用一块16 GB显存的Tesla V100 GPU即可完成训练，充分证明了所提框架的显著效率。