Existing deep learning-based shadow removal methods still produce images with shadow remnants. These shadow remnants typically exist in homogeneous regions with low-intensity values, making them untraceable in the existing image-to-image mapping paradigm. We observe that shadows mainly degrade images at the image-structure level (in which humans perceive object shapes and continuous colors). Hence, in this paper, we propose to remove shadows at the image structure level. Based on this idea, we propose a novel structure-informed shadow removal network (StructNet) to leverage the image-structure information to address the shadow remnant problem. Specifically, StructNet first reconstructs the structure information of the input image without shadows and then uses the restored shadow-free structure prior to guiding the image-level shadow removal. StructNet contains two main novel modules: (1) a mask-guided shadow-free extraction (MSFE) module to extract image structural features in a non-shadow-to-shadow directional manner, and (2) a multi-scale feature & residual aggregation (MFRA) module to leverage the shadow-free structure information to regularize feature consistency. In addition, we also propose to extend StructNet to exploit multi-level structure information (MStructNet), to further boost the shadow removal performance with minimum computational overheads. Extensive experiments on three shadow removal benchmarks demonstrate that our method outperforms existing shadow removal methods, and our StructNet can be integrated with existing methods to improve them further.

翻译：现有基于深度学习的阴影去除方法仍会产生含有阴影残留的图像。这些阴影残留通常存在于低亮度均匀区域，导致其在现有图像到图像映射范式中难以被追踪。我们观察到阴影主要在图像结构层面（人类在此层面感知物体形状与连续色彩）对图像造成退化。因此，本文提出在图像结构层面进行阴影去除。基于这一思路，我们设计了一种新颖的结构信息引导阴影去除网络（StructNet），利用图像结构信息解决阴影残留问题。具体而言，StructNet首先重建输入图像的无阴影结构信息，然后利用恢复的无阴影结构先验指导图像层面的阴影去除。StructNet包含两个核心创新模块：（1）掩膜引导的无阴影提取（MSFE）模块，用于以非阴影到阴影的方向性方式提取图像结构特征；（2）多尺度特征与残差聚合（MFRA）模块，利用无阴影结构信息规范特征一致性。此外，我们进一步扩展StructNet以利用多层级结构信息（MStructNet），在最小化计算开销的前提下提升阴影去除性能。在三个阴影去除基准上的大量实验表明，我们的方法优于现有阴影去除方法，且StructNet可集成至现有方法中进一步提升其性能。