Fully-supervised shadow removal methods achieve top restoration qualities on public datasets but still generate some shadow remnants. One of the reasons is the lack of large-scale shadow & shadow-free image pairs. Unsupervised methods can alleviate the issue but their restoration qualities are much lower than those of fully-supervised methods. In this work, we find that pretraining shadow removal networks on the image inpainting dataset can reduce the shadow remnants significantly: a naive encoder-decoder network gets competitive restoration quality w.r.t. the state-of-the-art methods via only 10% shadow & shadow-free image pairs. We further analyze the difference between networks with/without inpainting pretraining and observe that: inpainting pretraining enhances networks' capability of filling missed semantic information; shadow removal fine-tuning makes the networks know how to fill details of the shadow regions. Inspired by the above observations, we formulate shadow removal as a shadow-guided inpainting task to take advantage of the shadow removal and image inpainting. Specifically, we build a shadow-informed dynamic filtering network with two branches: the image inpainting branch takes the shadow-masked image as input while the second branch takes the shadow image as input and is to estimate dynamic kernels and offsets for the first branch to provide missing semantic information and details. The extensive experiments show that our method empowered with inpainting outperforms all state-of-the-art methods.

翻译：全监督阴影去除方法在公共数据集上取得了顶尖的恢复质量，但仍会产生一些阴影残留。原因之一在于缺乏大规模成对的阴影与无阴影图像。无监督方法可缓解该问题，但其恢复质量远低于全监督方法。本研究发现，在图像修复数据集上预训练阴影去除网络能显著减少阴影残留：一个简单的编码器-解码器网络仅需10%的阴影与无阴影图像对，即可获得与最先进方法相当的恢复质量。我们进一步分析了有无修复预训练的网络差异，观察到：修复预训练增强了网络填补缺失语义信息的能力；阴影去除微调使网络学会如何填充阴影区域的细节。受上述观察启发，我们将阴影去除公式化为阴影引导的修复任务，以充分利用阴影去除与图像修复的优势。具体而言，我们构建了一个阴影感知动态滤波网络，包含两个分支：图像修复分支以阴影掩蔽图像为输入，第二分支以阴影图像为输入，并为第一分支估计动态核与偏移量，以提供缺失的语义信息与细节。大量实验表明，结合修复能力的方法优于所有现有最先进方法。