Recently, Transformers have shown promising performance in various vision tasks. However, the high costs of global self-attention remain challenging for Transformers, especially for high-resolution vision tasks. Local self-attention runs attention computation within a limited region for the sake of efficiency, resulting in insufficient context modeling as their receptive fields are small. In this work, we introduce two new attention modules to enhance the global modeling capability of the hierarchical vision transformer, namely, random sampling windows (RS-Win) and important region windows (IR-Win). Specifically, RS-Win sample random image patches to compose the window, following a uniform distribution, i.e., the patches in RS-Win can come from any position in the image. IR-Win composes the window according to the weights of the image patches in the attention map. Notably, RS-Win is able to capture global information throughout the entire model, even in earlier, high-resolution stages. IR-Win enables the self-attention module to focus on important regions of the image and capture more informative features. Incorporated with these designs, RSIR-Win Transformer demonstrates competitive performance on common vision tasks.

翻译：最近，Transformer在各类视觉任务中展现出优异的性能。然而，全局自注意力机制的高计算成本仍构成挑战，尤其在高分辨率视觉任务中。局部自注意力为提升效率仅在有限区域内计算注意力，因其感受野较小导致上下文建模不足。本文提出两种新型注意力模块以增强层次化视觉Transformer的全局建模能力，即随机采样窗口（RS-Win）和重要区域窗口（IR-Win）。具体而言，RS-Win通过均匀分布随机选取图像块构成窗口，即窗口内图像块可来自图像任意位置；IR-Win则根据注意力图中图像块的权重构成窗口。值得注意的是，RS-Win能够在整个模型中（包括早期高分辨率阶段）捕获全局信息。IR-Win使自注意力模块聚焦于图像重要区域，提取更具信息量的特征。结合这些设计，RSIR-Win Transformer在常见视觉任务中展现出具有竞争力的性能。