Image restoration is a long-standing low-level vision problem, e.g., deblurring and deraining. In the process of image restoration, it is necessary to consider not only the spatial details and contextual information of restoration to ensure the quality, but also the system complexity. Although many methods have been able to guarantee the quality of image restoration, the system complexity of the state-of-the-art (SOTA) methods is increasing as well. Motivated by this, we present a mixed hierarchy network that can balance these competing goals. Our main proposal is a mixed hierarchy architecture, that progressively recovers contextual information and spatial details from degraded images while we design intra-blocks to reduce system complexity. Specifically, our model first learns the contextual information using encoder-decoder architectures, and then combines them with high-resolution branches that preserve spatial detail. In order to reduce the system complexity of this architecture for convenient analysis and comparison, we replace or remove the nonlinear activation function with multiplication and use a simple network structure. In addition, we replace spatial convolution with global self-attention for the middle block of encoder-decoder. The resulting tightly interlinked hierarchy architecture, named as MHNet, delivers strong performance gains on several image restoration tasks, including image deraining, and deblurring.

翻译：图像恢复是一个长期存在的低级视觉问题，例如去模糊和去雨。在图像恢复过程中，不仅需要考虑恢复的空间细节和上下文信息以保证质量，还需关注系统复杂度。尽管许多方法已能保证图像恢复质量，但现有最先进（SOTA）方法的系统复杂度也在不断增加。受此启发，我们提出了一种混合层次网络，能够平衡这些相互竞争的目标。我们的核心方案是一种混合层次架构，该架构从退化图像中逐步恢复上下文信息和空间细节，同时通过设计内嵌模块来降低系统复杂度。具体而言，模型首先利用编码器-解码器架构学习上下文信息，随后将其与保留空间细节的高分辨率分支相结合。为降低该架构的系统复杂度以方便分析与比较，我们采用乘法替代或移除非线性激活函数，并使用简单的网络结构。此外，我们将编码器-解码器的中间模块中的空间卷积替换为全局自注意力机制。由此形成的紧密耦合层次架构（命名为MHNet）在多项图像恢复任务（包括图像去雨和去模糊）中展现出强劲的性能提升。