Image enhancement algorithms are very useful for real world computer vision tasks where image resolution is often physically limited by the sensor size. While state-of-the-art deep neural networks show impressive results for image enhancement, they often struggle to enhance real-world images. In this work, we tackle a real-world setting: inpainting of images from Dunhuang caves. The Dunhuang dataset consists of murals, half of which suffer from corrosion and aging. These murals feature a range of rich content, such as Buddha statues, bodhisattvas, sponsors, architecture, dance, music, and decorative patterns designed by different artists spanning ten centuries, which makes manual restoration challenging. We modify two different existing methods (CAR, HINet) that are based upon state-of-the-art (SOTA) super resolution and deblurring networks. We show that those can successfully inpaint and enhance these deteriorated cave paintings. We further show that a novel combination of CAR and HINet, resulting in our proposed inpainting network (ARIN), is very robust to external noise, especially Gaussian noise. To this end, we present a quantitative and qualitative comparison of our proposed approach with existing SOTA networks and winners of the Dunhuang challenge. One of the proposed methods HINet) represents the new state of the art and outperforms the 1st place of the Dunhuang Challenge, while our combination ARIN, which is robust to noise, is comparable to the 1st place. We also present and discuss qualitative results showing the impact of our method for inpainting on Dunhuang cave images.
翻译:图像增强算法对于传感器尺寸物理限制图像分辨率的真实世界计算机视觉任务至关重要。尽管最先进的深度神经网络在图像增强方面展现出惊人效果,但在处理真实世界图像时仍面临挑战。本研究聚焦于真实世界场景:敦煌壁画图像修复。敦煌数据集包含大量壁画,其中半数存在腐蚀与老化问题。这些壁画涵盖十大世纪不同艺术家创作的丰富内容,如佛像、菩萨、供养人、建筑、舞蹈、音乐及装饰图案,使其人工修复极具挑战性。我们改进了两种基于最先进超分辨率与去模糊网络的方法(CAR、HINet),证明其可有效修复并增强这些受蚀壁画。进一步研究表明,通过创新性结合CAR与HINet形成的修复网络ARIN,对外部噪声(尤其是高斯噪声)具有极强鲁棒性。我们通过定量与定性比较,将所提方法与现有最先进网络及敦煌挑战赛获胜者进行对比。其中HINet方法达到新最优性能,超越敦煌挑战赛第一名;而具有噪声鲁棒性的组合方法ARIN与第一名性能相当。此外,我们展示并讨论了定性结果,证明该方法在敦煌壁画图像修复中的实际效果。