Image restoration is a long-standing low-level vision problem, e.g., deblurring and deraining. In the process of image restoration, it is necessary to consider not only the spatial details and contextual information of restoration to ensure the quality, but also the system complexity. Although many methods have been able to guarantee the quality of image restoration, the system complexity of the state-of-the-art (SOTA) methods is increasing as well. Motivated by this, we present a mixed hierarchy network that can balance these competing goals. Our main proposal is a mixed hierarchy architecture, that progressively recovers contextual information and spatial details from degraded images while we design intra-blocks to reduce system complexity. Specifically, our model first learns the contextual information using encoder-decoder architectures, and then combines them with high-resolution branches that preserve spatial detail. In order to reduce the system complexity of this architecture for convenient analysis and comparison, we replace or remove the nonlinear activation function with multiplication and use a simple network structure. In addition, we replace spatial convolution with global self-attention for the middle block of encoder-decoder. The resulting tightly interlinked hierarchy architecture, named as MHNet, delivers strong performance gains on several image restoration tasks, including image deraining, and deblurring.

翻译：图像恢复是一个长期的底层视觉问题，例如去模糊和去雨。在图像恢复过程中，不仅需要考虑恢复的空间细节和上下文信息以保证质量，还需要考虑系统复杂度。尽管许多方法已能保证图像恢复的质量，但最先进方法的系统复杂度也在不断增加。受此启发，我们提出了一种混合层次网络，能够平衡这些相互竞争的目标。我们的主要创新是一种混合层次架构，该架构从退化图像中逐步恢复上下文信息和空间细节，同时我们设计了内部块来降低系统复杂度。具体而言，我们的模型首先使用编码器-解码器架构学习上下文信息，然后将其与保留空间细节的高分辨率分支相结合。为了降低该架构的系统复杂度以便于分析和比较，我们使用乘法替代或移除非线性激活函数，并采用简单的网络结构。此外，我们用全局自注意力替换了编码器-解码器中间块的空间卷积。由此产生的紧密互联的层次架构（命名为MHNet）在多个图像恢复任务（包括图像去雨和去模糊）中取得了显著的性能提升。