Transformer-based image denoising methods have achieved encouraging results in the past year. However, it must uses linear operations to model long-range dependencies, which greatly increases model inference time and consumes GPU storage space. Compared with convolutional neural network-based methods, current Transformer-based image denoising methods cannot achieve a balance between performance improvement and resource consumption. In this paper, we propose an Efficient Wavelet Transformer (EWT) for image denoising. Specifically, we use Discrete Wavelet Transform (DWT) and Inverse Wavelet Transform (IWT) for downsampling and upsampling, respectively. This method can fully preserve the image features while reducing the image resolution, thereby greatly reducing the device resource consumption of the Transformer model. Furthermore, we propose a novel Dual-stream Feature Extraction Block (DFEB) to extract image features at different levels, which can further reduce model inference time and GPU memory usage. Experiments show that our method speeds up the original Transformer by more than 80%, reduces GPU memory usage by more than 60%, and achieves excellent denoising results. All code will be public.

翻译：基于Transformer的图像去噪方法在过去一年中取得了令人鼓舞的成果。然而，其必须利用线性运算来建模长距离依赖关系，这极大地增加了模型推理时间并消耗了GPU存储空间。与基于卷积神经网络的方法相比，当前基于Transformer的图像去噪方法无法在性能提升与资源消耗之间取得平衡。本文提出了一种高效小波Transformer（EWT）用于图像去噪。具体而言，我们分别采用离散小波变换（DWT）和逆小波变换（IWT）进行下采样和上采样。该方法能在降低图像分辨率的同时充分保留图像特征，从而大幅降低Transformer模型的设备资源消耗。此外，我们提出了一种新颖的双流特征提取模块（DFEB）来提取不同层次的图像特征，可进一步减少模型推理时间和GPU内存占用。实验表明，我们的方法将原始Transformer的推理速度提升超过80%，GPU内存占用降低超过60%，并取得了优异的去噪效果。所有代码将公开。