Multi-scale representations are crucial for semantic segmentation. The community has witnessed the flourish of semantic segmentation convolutional neural networks (CNN) exploiting multi-scale contextual information. Motivated by that the vision transformer (ViT) is powerful in image classification, some semantic segmentation ViTs are recently proposed, most of them attaining impressive results but at a cost of computational economy. In this paper, we succeed in introducing multi-scale representations into semantic segmentation ViT via window attention mechanism and further improves the performance and efficiency. To this end, we introduce large window attention which allows the local window to query a larger area of context window at only a little computation overhead. By regulating the ratio of the context area to the query area, we enable the $\textit{large window attention}$ to capture the contextual information at multiple scales. Moreover, the framework of spatial pyramid pooling is adopted to collaborate with $\textit{the large window attention}$, which presents a novel decoder named $\textbf{la}$rge $\textbf{win}$dow attention spatial pyramid pooling (LawinASPP) for semantic segmentation ViT. Our resulting ViT, Lawin Transformer, is composed of an efficient hierachical vision transformer (HVT) as encoder and a LawinASPP as decoder. The empirical results demonstrate that Lawin Transformer offers an improved efficiency compared to the existing method. Lawin Transformer further sets new state-of-the-art performance on Cityscapes (84.4% mIoU), ADE20K (56.2% mIoU) and COCO-Stuff datasets. The code will be released at https://github.com/yan-hao-tian/lawin

翻译：多尺度表示对语义分割至关重要。学界已见证了利用多尺度上下文信息的语义分割卷积神经网络（CNN）的蓬勃发展。受视觉Transformer（ViT）在图像分类中表现强大的启发，近期提出了一些语义分割ViT，其中多数取得了令人瞩目的结果，但以计算经济性为代价。本文成功通过窗口注意力机制将多尺度表示引入语义分割ViT，并进一步提升了性能与效率。为此，我们引入大窗口注意力，使局部窗口能够以极小的计算开销查询更大范围的上下文窗口。通过调整上下文区域与查询区域的比例，我们使$\textit{大窗口注意力}$能够捕捉多尺度的上下文信息。此外，采用空间金字塔池化框架与$\textit{大窗口注意力}$协同工作，由此提出一种名为$\textbf{大窗口注意力空间金字塔池化}$（LawinASPP）的新型解码器，用于语义分割ViT。我们最终的ViT——Lawin Transformer，由一个高效的分层视觉Transformer（HVT）作为编码器，以及一个LawinASPP作为解码器构成。实验结果表明，与现有方法相比，Lawin Transformer在效率上有所提升。Lawin Transformer进一步在Cityscapes（84.4% mIoU）、ADE20K（56.2% mIoU）和COCO-Stuff数据集上取得了新的最优性能。代码将发布于https://github.com/yan-hao-tian/lawin。