Unsupervised domain adaptation object detection(UDAOD) research on Detection Transformer(DETR) mainly focuses on feature alignment and existing methods can be divided into two kinds, each of which has its unresolved issues. One-stage feature alignment methods can easily lead to performance fluctuation and training stagnation. Two-stage feature alignment method based on mean teacher comprises a pretraining stage followed by a self-training stage, each facing problems in obtaining reliable pretrained model and achieving consistent performance gains. Methods mentioned above have not yet explore how to utilize the third related domain such as target-like domain to assist adaptation. To address these issues, we propose a two-stage framework named MTM, i.e. Mean Teacher-DETR with Masked Feature Alignment. In the pretraining stage, we utilize labeled target-like images produced by image style transfer to avoid performance fluctuation. In the self-training stage, we leverage unlabeled target images by pseudo labels based on mean teacher and propose a module called Object Queries Knowledge Transfer(OQKT) to ensure consistent performance gains of the student model. Most importantly, we propose masked feature alignment methods including Masked Domain Query-based Feature Alignment(MDQFA) and Masked Token-wise Feature Alignment(MTWFA) to alleviate domain shift in a more robust way, which not only prevent training stagnation and lead to a robust pretrained model in the pretraining stage, but also enhance the model's target performance in the self-training stage. Experiments on three challenging scenarios and a theoretical analysis verify the effectiveness of MTM.
翻译:无监督领域自适应目标检测(UDAOD)在检测Transformer(DETR)上的研究主要聚焦于特征对齐,现有方法可分为两类,每类均存在未解决的问题。单阶段特征对齐方法易导致性能波动与训练停滞。基于均值教师的两阶段特征对齐方法包含预训练阶段和自训练阶段,各自面临获取可靠预训练模型与实现一致性性能提升的难题。上述方法尚未探索如何利用第三方相关领域(如目标相似领域)辅助自适应。为解决这些问题,我们提出名为MTM的两阶段框架,即带掩码特征对齐的均值教师DETR。在预训练阶段,我们利用图像风格迁移生成带标签的目标相似图像以避免性能波动。在自训练阶段,我们基于均值教师通过伪标签利用无标签目标图像,并提出名为目标查询知识迁移(OQKT)的模块确保学生模型性能的持续提升。最重要的是,我们提出掩码特征对齐方法,包括基于掩码域查询的特征对齐(MDQFA)和掩码令牌级特征对齐(MTWFA),以更鲁棒的方式缓解领域偏移——这不仅在预训练阶段防止训练停滞并获得稳健的预训练模型,同时在自训练阶段增强模型的目标域性能。在三个具有挑战性场景上的实验及理论分析验证了MTM的有效性。