Feature Normalization (FN) is an important technique to help neural network training, which typically normalizes features across spatial dimensions. Most previous image inpainting methods apply FN in their networks without considering the impact of the corrupted regions of the input image on normalization, e.g. mean and variance shifts. In this work, we show that the mean and variance shifts caused by full-spatial FN limit the image inpainting network training and we propose a spatial region-wise normalization named Region Normalization (RN) to overcome the limitation. RN divides spatial pixels into different regions according to the input mask, and computes the mean and variance in each region for normalization. We develop two kinds of RN for our image inpainting network: (1) Basic RN (RN-B), which normalizes pixels from the corrupted and uncorrupted regions separately based on the original inpainting mask to solve the mean and variance shift problem; (2) Learnable RN (RN-L), which automatically detects potentially corrupted and uncorrupted regions for separate normalization, and performs global affine transformation to enhance their fusion. We apply RN-B in the early layers and RN-L in the latter layers of the network respectively. Experiments show that our method outperforms current state-of-the-art methods quantitatively and qualitatively. We further generalize RN to other inpainting networks and achieve consistent performance improvements. Our code is available at https://github.com/geekyutao/RN.

翻译：特征归一化（FN）是辅助神经网络训练的重要技术，通常沿空间维度对特征进行归一化。先前大多数图像修复方法在其网络中使用FN时，未考虑输入图像中损坏区域对归一化（如均值和方差偏移）的影响。本研究表明，全空间FN导致的均值和方差偏移会限制图像修复网络的训练，为此我们提出一种空间区域级归一化方法——区域归一化（RN）以克服这一局限。RN根据输入掩膜将空间像素划分为不同区域，并在每个区域内计算均值和方差进行归一化。我们为图像修复网络开发了两种RN：（1）基本RN（RN-B），基于原始修复掩膜分别对损坏和未损坏区域的像素独立归一化，以解决均值和方差偏移问题；（2）可学习RN（RN-L），自动检测潜在损坏和未损坏区域进行分离归一化，并通过全局仿射变换增强区域融合。我们在网络浅层使用RN-B，深层使用RN-L。实验表明，本方法在定量与定性指标上均超越现有最优方法。我们进一步将RN推广至其他修复网络，取得一致性能提升。代码开源于 https://github.com/geekyutao/RN。