Image restoration is a long-standing low-level vision problem, e.g., deblurring and deraining. In the process of image restoration, it is necessary to consider not only the spatial details and contextual information of restoration to ensure the quality, but also the system complexity. Although many methods have been able to guarantee the quality of image restoration, the system complexity of the state-of-the-art (SOTA) methods is increasing as well. Motivated by this, we present a mixed hierarchy network that can balance these competing goals. Our main proposal is a mixed hierarchy architecture, that progressively recovers contextual information and spatial details from degraded images while we design intra-blocks to reduce system complexity. Specifically, our model first learns the contextual information using encoder-decoder architectures, and then combines them with high-resolution branches that preserve spatial detail. In order to reduce the system complexity of this architecture for convenient analysis and comparison, we replace or remove the nonlinear activation function with multiplication and use a simple network structure. In addition, we replace spatial convolution with global self-attention for the middle block of encoder-decoder. The resulting tightly interlinked hierarchy architecture, named as MHNet, delivers strong performance gains on several image restoration tasks, including image deraining, and deblurring.

翻译：图像恢复是一个长期存在的低级视觉问题，例如去模糊和去雨。在图像恢复过程中，不仅要考虑恢复的空间细节和上下文信息以确保质量，还需要考虑系统复杂度。尽管许多方法已能保证图像恢复的质量，但最先进（SOTA）方法的系统复杂度也在不断增加。受此启发，我们提出了一种混合层次网络，能够在这些相互竞争的目标之间取得平衡。我们的主要方案是一种混合层次架构，它从退化图像中逐步恢复上下文信息和空间细节，同时我们设计了内部块来降低系统复杂度。具体而言，我们的模型首先利用编码器-解码器架构学习上下文信息，然后将其与保留空间细节的高分辨率分支相结合。为了降低该架构的系统复杂度以便于分析和比较，我们使用乘法替代或移除非线性激活函数，并采用简单的网络结构。此外，我们用全局自注意力替代编码器-解码器中间块的空间卷积。由此产生的紧密互连的层次架构（命名为MHNet）在多个图像恢复任务（包括图像去雨和去模糊）上展现出显著的性能提升。