Recently, the contrastive learning paradigm has achieved remarkable success in high-level tasks such as classification, detection, and segmentation. However, contrastive learning applied in low-level tasks, like image restoration, is limited, and its effectiveness is uncertain. This raises a question: Why does the contrastive learning paradigm not yield satisfactory results in image restoration? In this paper, we conduct in-depth analyses and propose three guidelines to address the above question. In addition, inspired by style transfer and based on contrastive learning, we propose a novel module for image restoration called \textbf{ConStyle}, which can be efficiently integrated into any U-Net structure network. By leveraging the flexibility of ConStyle, we develop a \textbf{general restoration network} for image restoration. ConStyle and the general restoration network together form an image restoration framework, namely \textbf{IRConStyle}. To demonstrate the capability and compatibility of ConStyle, we replace the general restoration network with transformer-based, CNN-based, and MLP-based networks, respectively. We perform extensive experiments on various image restoration tasks, including denoising, deblurring, deraining, and dehazing. The results on 19 benchmarks demonstrate that ConStyle can be integrated with any U-Net-based network and significantly enhance performance. For instance, ConStyle NAFNet significantly outperforms the original NAFNet on SOTS outdoor (dehazing) and Rain100H (deraining) datasets, with PSNR improvements of 4.16 dB and 3.58 dB with 85% fewer parameters.
翻译:近年来,对比学习范式在分类、检测和分割等高级任务中取得了显著成功。然而,对比学习在图像恢复等低级任务中的应用有限,其有效性尚不确定。这引出一个问题:为什么对比学习范式在图像恢复中未能取得令人满意的结果?本文通过深入分析,提出三条准则来解答上述问题。此外,受风格迁移启发并基于对比学习,我们提出一个名为\textbf{ConStyle}的新型图像恢复模块,该模块可高效集成到任何U-Net结构网络中。利用ConStyle的灵活性,我们开发了一个\textbf{通用恢复网络}用于图像恢复。ConStyle与通用恢复网络共同构成一个图像恢复框架,即\textbf{IRConStyle}。为展示ConStyle的能力与兼容性,我们分别用基于Transformer、CNN和MLP的网络替换通用恢复网络。我们在多种图像恢复任务(包括去噪、去模糊、去雨和去雾)上进行大量实验。在19个基准测试上的结果表明,ConStyle可与任何基于U-Net的网络集成,并显著提升性能。例如,ConStyle NAFNet在SOTS户外(去雾)和Rain100H(去雨)数据集上显著优于原始NAFNet,PSNR分别提升4.16 dB和3.58 dB,且参数减少85%。