Transformer is beneficial for image denoising tasks since it can model long-range dependencies to overcome the limitations presented by inductive convolutional biases. However, directly applying the transformer structure to remove noise is challenging because its complexity grows quadratically with the spatial resolution. In this paper, we propose an efficient Dual-branch Deformable Transformer (DDT) denoising network which captures both local and global interactions in parallel. We divide features with a fixed patch size and a fixed number of patches in local and global branches, respectively. In addition, we apply deformable attention operation in both branches, which helps the network focus on more important regions and further reduces computational complexity. We conduct extensive experiments on real-world and synthetic denoising tasks, and the proposed DDT achieves state-of-the-art performance with significantly fewer computational costs.

翻译：Transformer能够建模长距离依赖关系，克服归纳卷积偏置带来的局限性，因此对图像去噪任务十分有益。然而，直接应用Transformer结构去除噪声具有挑战性，因为其复杂度随空间分辨率呈二次方增长。本文提出一种高效的双分支可变形Transformer（DDT）去噪网络，能够并行捕获局部和全局交互。我们在局部和全局分支中分别采用固定补丁大小和固定补丁数量的特征划分方式。此外，我们在两个分支中均应用可变形注意力操作，这有助于网络聚焦于更重要的区域，并进一步降低计算复杂度。我们在真实世界和合成去噪任务上进行了大量实验，所提出的DDT在显著降低计算成本的同时取得了最先进的性能。