We present in this paper a new architecture, the Pattern Attention Transformer (PAT), that is composed of the new doughnut kernel. Compared with tokens in the NLP field, Transformer in computer vision has the problem of handling the high resolution of pixels in images. In ViT, an image is cut into square-shaped patches. As the follow-up of ViT, Swin Transformer proposes an additional step of shifting to decrease the existence of fixed boundaries, which also incurs 'two connected Swin Transformer blocks' as the minimum unit of the model. Inheriting the patch/window idea, our doughnut kernel enhances the design of patches further. It replaces the line-cut boundaries with two types of areas: sensor and updating, which is based on the comprehension of self-attention (named QKVA grid). The doughnut kernel also brings a new topic about the shape of kernels beyond square. To verify its performance on image classification, PAT is designed with Transformer blocks of regular octagon shape doughnut kernels. Its architecture is lighter: the minimum pattern attention layer is only one for each stage. Under similar complexity of computation, its performances on ImageNet 1K reach higher throughput (+10%) and surpass Swin Transformer (+0.1 acc1).

翻译：本文提出一种新型架构——图案注意力Transformer（PAT），其核心由新型甜甜圈核构成。与自然语言处理领域的标记（token）不同，计算机视觉中的Transformer面临处理图像高分辨率像素的难题。在ViT中，图像被切割为方形补丁。作为ViT的后续工作，Swin Transformer引入了额外的移位步骤以减少固定边界的存在，但这导致"两个相连的Swin Transformer块"成为模型的最小单元。继承补丁/窗口思想，我们的甜甜圈核进一步增强了补丁设计：它将线性切割边界替换为两类区域——感知区与更新区（基于自注意力机制的理解，即QKVA网格）。甜甜圈核还引发了关于超越方形核形状的新课题。为验证其在图像分类中的性能，我们设计了采用正八边形甜甜圈核的Transformer块。其架构更为轻量：每个阶段仅需一个最小图案注意力层。在相似计算复杂度下，该模型在ImageNet 1K数据集上实现了更高吞吐量（+10%），并在top-1准确率上超越Swin Transformer（+0.1%）。