The under-display camera (UDC) provides consumers with a full-screen visual experience without any obstruction due to notches or punched holes. However, the semi-transparent nature of the display inevitably introduces the severe degradation into UDC images. In this work, we address the UDC image restoration problem with the specific consideration of the scattering effect caused by the display. We explicitly model the scattering effect by treating the display as a piece of homogeneous scattering medium. With the physical model of the scattering effect, we improve the image formation pipeline for the image synthesis to construct a realistic UDC dataset with ground truths. To suppress the scattering effect for the eventual UDC image recovery, a two-branch restoration network is designed. More specifically, the scattering branch leverages global modeling capabilities of the channel-wise self-attention to estimate parameters of the scattering effect from degraded images. While the image branch exploits the local representation advantage of CNN to recover clear scenes, implicitly guided by the scattering branch. Extensive experiments are conducted on both real-world and synthesized data, demonstrating the superiority of the proposed method over the state-of-the-art UDC restoration techniques. The source code and dataset are available at \url{https://github.com/NamecantbeNULL/SRUDC}.
翻译:屏下相机(UDC)为消费者提供了无 notch 或打孔遮挡的全屏视觉体验。然而,显示屏的半透明特性不可避免地为 UDC 图像引入了严重退化。本文针对 UDC 图像复原问题,特别考虑了由显示屏引起的散射效应。我们将显示屏视为均匀散射介质,对散射效应进行了显式建模。基于散射效应的物理模型,我们改进了图像合成管线以构建具有真实参考的 UDC 数据集。为抑制散射效应以实现最终的 UDC 图像恢复,设计了一种双分支复原网络:散射分支利用通道自注意力的全局建模能力,从退化图像中估计散射效应的参数;图像分支则借助 CNN 的局部表示优势,在散射分支的隐式引导下恢复清晰场景。在真实数据与合成数据上的大量实验表明,所提方法优于当前最先进的 UDC 复原技术。源代码与数据集已发布于 \url{https://github.com/NamecantbeNULL/SRUDC}。