Positional encoding is important for vision transformer (ViT) to capture the spatial structure of the input image. General effectiveness has been proven in ViT. In our work we propose to train ViT to recognize the positional label of patches of the input image, this apparently simple task actually yields a meaningful self-supervisory task. Based on previous work on ViT positional encoding, we propose two positional labels dedicated to 2D images including absolute position and relative position. Our positional labels can be easily plugged into various current ViT variants. It can work in two ways: (a) As an auxiliary training target for vanilla ViT (e.g., ViT-B and Swin-B) for better performance. (b) Combine the self-supervised ViT (e.g., MAE) to provide a more powerful self-supervised signal for semantic feature learning. Experiments demonstrate that with the proposed self-supervised methods, ViT-B and Swin-B gain improvements of 1.20% (top-1 Acc) and 0.74% (top-1 Acc) on ImageNet, respectively, and 6.15% and 1.14% improvement on Mini-ImageNet.

翻译：位置编码对于视觉Transformer（ViT）捕捉输入图像的空间结构至关重要。其通用有效性已在ViT中得到验证。在本工作中，我们提出训练ViT识别输入图像补丁的位置标签，这一看似简单的任务实际上产生了有意义的自监督任务。基于先前ViT位置编码的研究，我们提出了两种专用于二维图像的绝对位置和相对位置标签。我们的位置标签可轻松集成到当前多种ViT变体中，并以两种方式工作：(a) 作为vanilla ViT（如ViT-B和Swin-B）的辅助训练目标，以提升性能；(b) 结合自监督ViT（如MAE），为语义特征学习提供更强的自监督信号。实验表明，采用所提出的自监督方法，ViT-B和Swin-B在ImageNet上分别获得1.20%（top-1准确率）和0.74%的提升，在Mini-ImageNet上则分别提升6.15%和1.14%。