Recently, Transformers have shown promising performance in various vision tasks. However, the high costs of global self-attention remain challenging for Transformers, especially for high-resolution vision tasks. Local self-attention runs attention computation within a limited region for the sake of efficiency, resulting in insufficient context modeling as their receptive fields are small. In this work, we introduce two new attention modules to enhance the global modeling capability of the hierarchical vision transformer, namely, random sampling windows (RS-Win) and important region windows (IR-Win). Specifically, RS-Win sample random image patches to compose the window, following a uniform distribution, i.e., the patches in RS-Win can come from any position in the image. IR-Win composes the window according to the weights of the image patches in the attention map. Notably, RS-Win is able to capture global information throughout the entire model, even in earlier, high-resolution stages. IR-Win enables the self-attention module to focus on important regions of the image and capture more informative features. Incorporated with these designs, RSIR-Win Transformer demonstrates competitive performance on common vision tasks.

翻译：近期，Transformer在各类视觉任务中展现出卓越性能。然而，全局自注意力机制的高昂成本仍是其面临的挑战，尤其在高分辨率视觉任务中。局部自注意力为提升效率仅在有限区域内进行注意力计算，导致感受野较小，上下文建模能力不足。本文提出两种新型注意力模块以增强分层视觉Transformer的全局建模能力：随机采样窗口（RS-Win）与重要区域窗口（IR-Win）。具体而言，RS-Win按照均匀分布随机采样图像块构成窗口，即窗口内图像块可源自图像任意位置；IR-Win则根据注意力图中图像块的权重构建窗口。值得注意的是，RS-Win即使在早期高分辨率阶段也能捕获全局信息，而IR-Win使自注意力模块聚焦于图像重要区域并提取更具判别力的特征。结合这些设计，RSIR-Win Transformer在常见视觉任务中展现出竞争性能。