We present the Group Propagation Vision Transformer (GPViT): a novel nonhierarchical (i.e. non-pyramidal) transformer model designed for general visual recognition with high-resolution features. High-resolution features (or tokens) are a natural fit for tasks that involve perceiving fine-grained details such as detection and segmentation, but exchanging global information between these features is expensive in memory and computation because of the way self-attention scales. We provide a highly efficient alternative Group Propagation Block (GP Block) to exchange global information. In each GP Block, features are first grouped together by a fixed number of learnable group tokens; we then perform Group Propagation where global information is exchanged between the grouped features; finally, global information in the updated grouped features is returned back to the image features through a transformer decoder. We evaluate GPViT on a variety of visual recognition tasks including image classification, semantic segmentation, object detection, and instance segmentation. Our method achieves significant performance gains over previous works across all tasks, especially on tasks that require highresolution outputs, for example, our GPViT-L3 outperforms Swin Transformer-B by 2.0 mIoU on ADE20K semantic segmentation with only half as many parameters. Project page: chenhongyiyang.com/projects/GPViT/GPViT

翻译：我们提出分组传播视觉Transformer（GPViT）：一种新颖的非层级（即非金字塔结构）Transformer模型，专为利用高分辨率特征进行通用视觉识别任务而设计。高分辨率特征（或令牌）天然适用于需要感知细粒度细节的任务（如检测与分割），但由于自注意力机制的缩放特性，在这些特征间交换全局信息在内存和计算上代价高昂。我们提出了一种高效替代方案——分组传播块（GP Block）来实现全局信息交换。在每个GP块中，特征首先通过固定数量的可学习分组令牌进行聚类；随后执行分组传播操作，在聚类特征间完成全局信息交换；最后通过Transformer解码器将更新后的聚类特征中的全局信息回传至图像特征。我们在图像分类、语义分割、目标检测和实例分割等多种视觉识别任务上评估了GPViT。该方法在所有任务上均显著优于先前工作，尤其在需高分辨率输出的任务中表现突出：例如，我们的GPViT-L3在参数量仅为Swin Transformer-B一半的情况下，在ADE20K语义分割任务上仍以2.0 mIoU领先。项目页面：chenhongyiyang.com/projects/GPViT/GPViT