This paper presents a new adversarial training framework for image inpainting with segmentation confusion adversarial training (SCAT) and contrastive learning. SCAT plays an adversarial game between an inpainting generator and a segmentation network, which provides pixel-level local training signals and can adapt to images with free-form holes. By combining SCAT with standard global adversarial training, the new adversarial training framework exhibits the following three advantages simultaneously: (1) the global consistency of the repaired image, (2) the local fine texture details of the repaired image, and (3) the flexibility of handling images with free-form holes. Moreover, we propose the textural and semantic contrastive learning losses to stabilize and improve our inpainting model's training by exploiting the feature representation space of the discriminator, in which the inpainting images are pulled closer to the ground truth images but pushed farther from the corrupted images. The proposed contrastive losses better guide the repaired images to move from the corrupted image data points to the real image data points in the feature representation space, resulting in more realistic completed images. We conduct extensive experiments on two benchmark datasets, demonstrating our model's effectiveness and superiority both qualitatively and quantitatively.

翻译：本文提出了一种融合分割混淆对抗训练（SCAT）与对比学习的图像修复新对抗训练框架。SCAT通过在修复生成器与分割网络之间构建对抗博弈，提供像素级局部训练信号，并能够适应具有自由形式孔洞的图像。通过将SCAT与标准全局对抗训练相结合，该新型对抗训练框架同时展现出以下三个优势：（1）修复图像的全局一致性，（2）修复图像局部的精细纹理细节，（3）处理自由形式孔洞图像的灵活性。此外，我们提出纹理与语义对比学习损失，通过利用判别器的特征表示空间，将修复图像拉近至真实图像的同时推离受损图像，从而稳定并提升修复模型的训练效果。所提出的对比损失能够更好地引导修复图像在特征表示空间中从受损图像数据点向真实图像数据点移动，从而生成更逼真的完整图像。我们在两个基准数据集上进行了广泛实验，从定性和定量两个角度证明了模型的有效性与优越性。