Visual object tracking often employs a multi-stage pipeline of feature extraction, target information integration, and bounding box estimation. To simplify this pipeline and unify the process of feature extraction and target information integration, in this paper, we present a compact tracking framework, termed as MixFormer, built upon transformers. Our core design is to utilize the flexibility of attention operations, and propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration. This synchronous modeling scheme allows to extract target-specific discriminative features and perform extensive communication between target and search area. Based on MAM, we build our MixFormer trackers simply by stacking multiple MAMs and placing a localization head on top. Specifically, we instantiate two types of MixFormer trackers, a hierarchical tracker MixCvT, and a non-hierarchical tracker MixViT. For these two trackers, we investigate a series of pre-training methods and uncover the different behaviors between supervised pre-training and self-supervised pre-training in our MixFormer trackers. We also extend the masked pre-training to our MixFormer trackers and design the competitive TrackMAE pre-training technique. Finally, to handle multiple target templates during online tracking, we devise an asymmetric attention scheme in MAM to reduce computational cost, and propose an effective score prediction module to select high-quality templates. Our MixFormer trackers set a new state-of-the-art performance on seven tracking benchmarks, including LaSOT, TrackingNet, VOT2020, GOT-10k, OTB100 and UAV123. In particular, our MixViT-L achieves AUC score of 73.3% on LaSOT, 86.1% on TrackingNet, EAO of 0.584 on VOT2020, and AO of 75.7% on GOT-10k. Code and trained models are publicly available at https://github.com/MCG-NJU/MixFormer.

翻译：视觉目标跟踪通常采用特征提取、目标信息整合与边界框估计的多阶段流程。为简化该流程并统一特征提取与目标信息整合过程，本文提出一种基于Transformer的紧凑型跟踪框架——MixFormer。其核心设计在于利用注意力操作的灵活性，提出混合注意力模块（MAM）以同步实现特征提取与目标信息整合。该同步建模机制能提取目标特定的判别性特征，并促进目标区域与搜索区域间的充分交互。基于MAM，我们通过堆叠多个MAM并在顶部设置定位头，构建了MixFormer跟踪器。具体地，我们实现了两种类型：分层跟踪器MixCvT与非分层跟踪器MixViT。针对这两类跟踪器，我们系统研究了多种预训练方法，揭示了有监督预训练与自监督预训练在MixFormer跟踪器中的差异性行为。同时，我们将掩码预训练扩展至MixFormer跟踪器，设计了具有竞争力的TrackMAE预训练技术。最后，为处理在线跟踪中的多目标模板问题，我们在MAM中设计了非对称注意力机制以降低计算开销，并提出有效的分数预测模块用于筛选高质量模板。我们的MixFormer跟踪器在LaSOT、TrackingNet、VOT2020、GOT-10k、OTB100及UAV123等七个跟踪基准上均创下最新最优性能。其中，MixViT-L在LaSOT上达到73.3%的AUC分数，在TrackingNet上达86.1%，在VOT2020上EAO为0.584，在GOT-10k上AO为75.7%。代码与训练模型已公开于https://github.com/MCG-NJU/MixFormer。