Image restoration is a long-standing low-level vision problem, e.g., deblurring and deraining. In the process of image restoration, it is necessary to consider not only the spatial details and contextual information of restoration to ensure the quality, but also the system complexity. Although many methods have been able to guarantee the quality of image restoration, the system complexity of the state-of-the-art (SOTA) methods is increasing as well. Motivated by this, we present a mixed hierarchy network that can balance these competing goals. Our main proposal is a mixed hierarchy architecture, that progressively recovers contextual information and spatial details from degraded images while we design intra-blocks to reduce system complexity. Specifically, our model first learns the contextual information using encoder-decoder architectures, and then combines them with high-resolution branches that preserve spatial detail. In order to reduce the system complexity of this architecture for convenient analysis and comparison, we replace or remove the nonlinear activation function with multiplication and use a simple network structure. In addition, we replace spatial convolution with global self-attention for the middle block of encoder-decoder. The resulting tightly interlinked hierarchy architecture, named as MHNet, delivers strong performance gains on several image restoration tasks, including image deraining, and deblurring.

翻译：图像恢复是一个长期存在的底层视觉问题，例如去模糊和去雨。在图像恢复过程中，不仅需要考虑恢复的空间细节和上下文信息以保证质量，还需要考虑系统复杂度。尽管许多方法已经能够保证图像恢复的质量，但当前最先进方法的系统复杂度也在增加。受此启发，我们提出了一种混合层次网络，能够在这些相互竞争的目标之间取得平衡。我们的主要创新是混合层次架构，它从退化图像中逐步恢复上下文信息和空间细节，同时我们设计了内部模块以降低系统复杂度。具体而言，我们的模型首先利用编码器-解码器结构学习上下文信息，然后将其与保留空间细节的高分辨率分支相结合。为了降低该架构的系统复杂度以便于分析和比较，我们使用乘法运算替代或移除非线性激活函数，并采用简单的网络结构。此外，我们将编码器-解码器的中间模块的空间卷积替换为全局自注意力机制。由此产生的紧密互联的层次架构，命名为MHNet，在多个图像恢复任务（包括图像去雨和去模糊）中展现出强大的性能提升。