In the field of Class Incremental Object Detection (CIOD), creating models that can continuously learn like humans is a major challenge. Pseudo-labeling methods, although initially powerful, struggle with multi-scenario incremental learning due to their tendency to forget past knowledge. To overcome this, we introduce a new approach called Vision-Language Model assisted Pseudo-Labeling (VLM-PL). This technique uses Vision-Language Model (VLM) to verify the correctness of pseudo ground-truths (GTs) without requiring additional model training. VLM-PL starts by deriving pseudo GTs from a pre-trained detector. Then, we generate custom queries for each pseudo GT using carefully designed prompt templates that combine image and text features. This allows the VLM to classify the correctness through its responses. Furthermore, VLM-PL integrates refined pseudo and real GTs from upcoming training, effectively combining new and old knowledge. Extensive experiments conducted on the Pascal VOC and MS COCO datasets not only highlight VLM-PL's exceptional performance in multi-scenario but also illuminate its effectiveness in dual-scenario by achieving state-of-the-art results in both.
翻译:在类增量目标检测(CIOD)领域,构建像人类一样持续学习的模型是一项重大挑战。伪标注方法虽然初始阶段表现强劲,但由于其易遗忘旧知识的特性,在应对多场景增量学习时面临困难。为解决此问题,我们提出一种创新方法——视觉-语言模型辅助伪标注(VLM-PL)。该技术利用视觉-语言模型(VLM)验证伪真值(GTs)的正确性,无需额外模型训练。VLM-PL首先从预训练检测器中推导出伪GTs,然后针对每个伪GT,通过精心设计的提示模板(融合图像与文本特征)生成定制化查询,使VLM能够依据其响应分类正确性。此外,VLM-PL整合后续训练中经优化的伪GTs与真实GTs,有效融合新旧知识。在Pascal VOC和MS COCO数据集上进行的大量实验不仅凸显了VLM-PL在多场景下的卓越性能,还揭示了其在双场景中的有效性——在这两种场景下均实现了最先进的结果。