Vision Transformer (ViT) has achieved remarkable performance in computer vision. However, positional encoding in ViT makes it substantially difficult to learn the intrinsic equivariance in data. Initial attempts have been made on designing equivariant ViT but are proved defective in some cases in this paper. To address this issue, we design a Group Equivariant Vision Transformer (GE-ViT) via a novel, effective positional encoding operator. We prove that GE-ViT meets all the theoretical requirements of an equivariant neural network. Comprehensive experiments are conducted on standard benchmark datasets, demonstrating that GE-ViT significantly outperforms non-equivariant self-attention networks. The code is available at https://github.com/ZJUCDSYangKaifan/GEVit.

翻译：视觉Transformer（Vision Transformer, ViT）在计算机视觉领域取得了显著性能。然而，ViT中的位置编码使得学习数据内在的等变性变得十分困难。尽管已有研究初步尝试设计等变ViT，但本文证明这些方法在某些情况下存在缺陷。为解决这一问题，我们通过一种新颖且有效的位置编码算子，设计了一种群等变视觉Transformer（Group Equivariant Vision Transformer, GE-ViT）。我们证明GE-ViT满足等变神经网络的所有理论要求。在标准基准数据集上进行的全面实验表明，GE-ViT的性能显著优于非等变的自注意力网络。代码已开源在https://github.com/ZJUCDSYangKaifan/GEVit。