The paradigm of large-scale pre-training followed by downstream fine-tuning has been widely employed in various object detection algorithms. In this paper, we reveal discrepancies in data, model, and task between the pre-training and fine-tuning procedure in existing practices, which implicitly limit the detector's performance, generalization ability, and convergence speed. To this end, we propose AlignDet, a unified pre-training framework that can be adapted to various existing detectors to alleviate the discrepancies. AlignDet decouples the pre-training process into two stages, i.e., image-domain and box-domain pre-training. The image-domain pre-training optimizes the detection backbone to capture holistic visual abstraction, and box-domain pre-training learns instance-level semantics and task-aware concepts to initialize the parts out of the backbone. By incorporating the self-supervised pre-trained backbones, we can pre-train all modules for various detectors in an unsupervised paradigm. As depicted in Figure 1, extensive experiments demonstrate that AlignDet can achieve significant improvements across diverse protocols, such as detection algorithm, model backbone, data setting, and training schedule. For example, AlignDet improves FCOS by 5.3 mAP, RetinaNet by 2.1 mAP, Faster R-CNN by 3.3 mAP, and DETR by 2.3 mAP under fewer epochs.

翻译：大规模预训练后接下游微调的范式已被广泛应用于各类目标检测算法中。本文揭示了现有实践中预训练与微调流程在数据、模型及任务层面存在的差异，这些差异隐式限制了检测器的性能、泛化能力及收敛速度。为此，我们提出AlignDet——一种统一的预训练框架，可适配现有多种检测器以缓解上述差异。AlignDet将预训练过程解耦为两个阶段：图像域预训练和框域预训练。图像域预训练优化检测骨干网络以捕捉整体视觉抽象特征，框域预训练则学习实例级语义与任务感知概念，从而初始化骨干网络之外的模块。通过结合自监督预训练骨干网络，我们能够以无监督范式预训练各类检测器的所有模块。如图1所示，大量实验表明，AlignDet在检测算法、模型骨干网络、数据设置及训练策略等多样化协议下均能实现显著改进。例如，在更少训练轮次下，AlignDet使FCOS提升5.3 mAP、RetinaNet提升2.1 mAP、Faster R-CNN提升3.3 mAP、DETR提升2.3 mAP。