Image inpainting involves filling missing areas of a corrupted image. Despite impressive results have been achieved recently, restoring images with both vivid textures and reasonable structures remains a significant challenge. Previous methods have primarily addressed regular textures while disregarding holistic structures due to the limited receptive fields of Convolutional Neural Networks (CNNs). To this end, we study learning a Zero-initialized residual addition based Incremental Transformer on Structural priors (ZITS++), an improved model upon our conference work, ZITS. Specifically, given one corrupt image, we present the Transformer Structure Restorer (TSR) module to restore holistic structural priors at low image resolution, which are further upsampled by Simple Structure Upsampler (SSU) module to higher image resolution. To recover image texture details, we use the Fourier CNN Texture Restoration (FTR) module, which is strengthened by Fourier and large-kernel attention convolutions. Furthermore, to enhance the FTR, the upsampled structural priors from TSR are further processed by Structure Feature Encoder (SFE) and optimized with the Zero-initialized Residual Addition (ZeroRA) incrementally. Besides, a new masking positional encoding is proposed to encode the large irregular masks. Compared with ZITS, ZITS++ improves the FTR's stability and inpainting ability with several techniques. More importantly, we comprehensively explore the effects of various image priors for inpainting and investigate how to utilize them to address high-resolution image inpainting with extensive experiments. This investigation is orthogonal to most inpainting approaches and can thus significantly benefit the community. Codes and models will be released in https://github.com/ewrfcas/ZITS-PlusPlus.

翻译：图像修复涉及填充受损图像中的缺失区域。尽管近期已取得显著成果，但恢复兼具生动纹理与合理结构的图像仍是一大挑战。现有方法主要处理规则纹理而忽视整体结构，这是由于卷积神经网络（CNN）的有限感受野所致。为此，我们研究学习一种基于零初始化残差相加的增量式Transformer结构先验方法（ZITS++），该模型是我们会议工作ZITS的改进版本。具体而言，针对给定受损图像，我们提出Transformer结构恢复器（TSR）模块，在低图像分辨率下恢复整体结构先验，并通过简单结构上采样器（SSU）模块将其上采样至更高分辨率。为恢复图像纹理细节，我们使用傅里叶CNN纹理恢复（FTR）模块，该模块通过傅里叶变换和大核注意力卷积得到增强。此外，为强化FTR，来自TSR的上采样结构先验经结构特征编码器（SFE）进一步处理后，通过零初始化残差相加（ZeroRA）增量式优化。同时，提出一种新型掩码位置编码以编码大型不规则掩码。相较于ZITS，ZITS++通过多项技术提升了FTR的稳定性与修复能力。更重要的是，我们全面探索了多种图像先验对修复的影响，并通过大量实验研究如何利用它们解决高分辨率图像修复问题。此研究与大多数修复方法正交，因此能显著惠及学术界。代码与模型将在https://github.com/ewrfcas/ZITS-PlusPlus发布。