Transformers have exhibited promising performance in computer vision tasks including image super-resolution (SR). However, popular transformer-based SR methods often employ window self-attention with quadratic computational complexity to window sizes, resulting in fixed small windows with limited receptive fields. In this paper, we present a general strategy to convert transformer-based SR networks to hierarchical transformers (HiT-SR), boosting SR performance with multi-scale features while maintaining an efficient design. Specifically, we first replace the commonly used fixed small windows with expanding hierarchical windows to aggregate features at different scales and establish long-range dependencies. Considering the intensive computation required for large windows, we further design a spatial-channel correlation method with linear complexity to window sizes, efficiently gathering spatial and channel information from hierarchical windows. Extensive experiments verify the effectiveness and efficiency of our HiT-SR, and our improved versions of SwinIR-Light, SwinIR-NG, and SRFormer-Light yield state-of-the-art SR results with fewer parameters, FLOPs, and faster speeds ($\sim7\times$).

翻译：Transformer在包括图像超分辨率（SR）在内的计算机视觉任务中已展现出有前景的性能。然而，当前主流的基于Transformer的SR方法通常采用具有相对于窗口尺寸呈二次计算复杂度的窗口自注意力机制，这导致其使用固定的小窗口且感受野有限。本文提出一种通用策略，可将基于Transformer的SR网络转换为分层Transformer（HiT-SR），在保持高效设计的同时，利用多尺度特征提升SR性能。具体而言，我们首先将常用的固定小窗口替换为扩展的分层窗口，以聚合不同尺度的特征并建立长程依赖关系。考虑到大窗口所需的密集计算，我们进一步设计了一种具有相对于窗口尺寸呈线性复杂度的空间-通道关联方法，从而高效地从分层窗口中收集空间与通道信息。大量实验验证了我们HiT-SR方法的有效性与高效性，我们对SwinIR-Light、SwinIR-NG和SRFormer-Light的改进版本以更少的参数量、更低的FLOPs和更快的速度（约7倍）取得了最先进的SR结果。