Visual object tracking often employs a multi-stage pipeline of feature extraction, target information integration, and bounding box estimation. To simplify this pipeline and unify the process of feature extraction and target information integration, in this paper, we present a compact tracking framework, termed as MixFormer, built upon transformers. Our core design is to utilize the flexibility of attention operations, and propose a Mixed Attention Module (MAM) for simultaneous feature extraction and target information integration. This synchronous modeling scheme allows to extract target-specific discriminative features and perform extensive communication between target and search area. Based on MAM, we build our MixFormer trackers simply by stacking multiple MAMs and placing a localization head on top. Specifically, we instantiate two types of MixFormer trackers, a hierarchical tracker MixCvT, and a non-hierarchical tracker MixViT. For these two trackers, we investigate a series of pre-training methods and uncover the different behaviors between supervised pre-training and self-supervised pre-training in our MixFormer trackers. We also extend the masked pre-training to our MixFormer trackers and design the competitive TrackMAE pre-training technique. Finally, to handle multiple target templates during online tracking, we devise an asymmetric attention scheme in MAM to reduce computational cost, and propose an effective score prediction module to select high-quality templates. Our MixFormer trackers set a new state-of-the-art performance on seven tracking benchmarks, including LaSOT, TrackingNet, VOT2020, GOT-10k, OTB100 and UAV123. In particular, our MixViT-L achieves AUC score of 73.3% on LaSOT, 86.1% on TrackingNet, EAO of 0.584 on VOT2020, and AO of 75.7% on GOT-10k. Code and trained models will be made available at https://github.com/MCG-NJU/MixFormer.

翻译：视觉目标跟踪通常采用特征提取、目标信息整合和边界框估计的多阶段流程。为简化这一流程并统一特征提取与目标信息整合过程，本文提出了一种基于Transformer的紧凑跟踪框架——MixFormer。我们的核心设计在于利用注意力操作的灵活性，提出混合注意力模块（MAM），实现特征提取与目标信息整合的同步进行。这种同步建模方案能够提取目标特定的判别性特征，并在目标与搜索区域间进行充分的信息交互。基于MAM，我们仅需堆叠多个MAM并在顶部放置定位头即可构建MixFormer跟踪器。具体而言，我们实现了两种MixFormer跟踪器：层次化跟踪器MixCvT和非层次化跟踪器MixViT。针对这两种跟踪器，我们研究了一系列预训练方法，揭示了有监督预训练与自监督预训练在MixFormer跟踪器中的不同表现。此外，我们将掩码预训练扩展至MixFormer跟踪器，并设计了具有竞争力的TrackMAE预训练技术。最后，为应对在线跟踪中的多目标模板问题，我们在MAM中设计了非对称注意力机制以降低计算成本，并提出有效的分数预测模块来筛选高质量模板。我们的MixFormer跟踪器在七个跟踪基准（包括LaSOT、TrackingNet、VOT2020、GOT-10k、OTB100和UAV123）上刷新了最优性能。其中，我们的MixViT-L在LaSOT上达到73.3%的AUC分数，在TrackingNet上为86.1%，在VOT2020上EAO为0.584，在GOT-10k上AO为75.7%。代码与预训练模型将在https://github.com/MCG-NJU/MixFormer 公开。