In this paper, we present a new sequence-to-sequence learning framework for visual tracking, dubbed SeqTrack. It casts visual tracking as a sequence generation problem, which predicts object bounding boxes in an autoregressive fashion. This is different from prior Siamese trackers and transformer trackers, which rely on designing complicated head networks, such as classification and regression heads. SeqTrack only adopts a simple encoder-decoder transformer architecture. The encoder extracts visual features with a bidirectional transformer, while the decoder generates a sequence of bounding box values autoregressively with a causal transformer. The loss function is a plain cross-entropy. Such a sequence learning paradigm not only simplifies tracking framework, but also achieves competitive performance on benchmarks. For instance, SeqTrack gets 72.5% AUC on LaSOT, establishing a new state-of-the-art performance. Code and models are available at here.

翻译：本文提出了一种新的序列到序列学习框架用于视觉跟踪，称为SeqTrack。该框架将视觉跟踪转化为序列生成问题，以自回归方式预测目标边界框。这与以往依赖设计复杂头部网络（如分类头和回归头）的孪生网络跟踪器和Transformer跟踪器不同。SeqTrack仅采用简单的编码器-解码器Transformer架构：编码器通过双向Transformer提取视觉特征，解码器则通过因果Transformer以自回归方式生成边界框值序列。损失函数采用简单的交叉熵。这种序列学习范式不仅简化了跟踪框架，还在基准测试中取得了具有竞争力的性能。例如，SeqTrack在LaSOT上达到72.5%的AUC，刷新了最先进性能记录。代码与模型已开源（此处附链接）。