The Yongle Palace murals, as valuable cultural heritage, have suffered varying degrees of damage, making their restoration of significant importance. However, the giant size and unique data of Yongle Palace murals present challenges for existing deep-learning based restoration methods: 1) The distinctive style introduces domain bias in traditional transfer learning-based restoration methods, while the scarcity of mural data further limits the applicability of these methods. 2) Additionally, the giant size of these murals results in a wider range of defect types and sizes, necessitating models with greater adaptability. Consequently, there is a lack of focus on deep learning-based restoration methods for the unique giant murals of Yongle Palace. Here, a 3M-Hybrid model is proposed to address these challenges. Firstly, based on the characteristic that the mural data frequency is prominent in the distribution of low and high frequency features, high and low frequency features are separately abstracted for complementary learning. Furthermore, we integrate a pre-trained Vision Transformer model (VIT) into the CNN module, allowing us to leverage the benefits of a large model while mitigating domain bias. Secondly, we mitigate seam and structural distortion issues resulting from the restoration of large defects by employing a multi-scale and multi-perspective strategy, including data segmentation and fusion. Experimental results demonstrate the efficacy of our proposed model. In regular-sized mural restoration, it improves SSIM and PSNR by 14.61% and 4.73%, respectively, compared to the best model among four representative CNN models. Additionally, it achieves favorable results in the final restoration of giant murals.

翻译：永乐宫壁画作为珍贵文化遗产已遭受不同程度的损坏，其修复工作具有重要意义。然而，永乐宫壁画的巨型尺寸和独特数据给现有基于深度学习的修复方法带来挑战：1）独特风格在传统迁移学习修复方法中引发领域偏差，而壁画数据稀缺进一步限制了这些方法的适用性；2）巨型尺寸导致缺陷类型与规模范围更广，要求模型具备更强的自适应能力。因此，针对永乐宫独特巨幅壁画的深度学习修复方法研究尚存空白。本文提出一种3M混合模型应对上述挑战。首先，基于壁画数据频谱在低频与高频特征分布中具有显著性的特点，分别抽象高低频特征以实现互补学习。同时，将预训练视觉Transformer模型（VIT）集成至CNN模块，在利用大模型优势的同时缓解领域偏差。其次，采用包含数据分割与融合的多尺度多视角策略，缓解大尺寸缺陷修复导致的接缝与结构畸变问题。实验证明所提模型的有效性：在常规尺寸壁画修复中，相较四个代表性CNN模型中的最优模型，SSIM与PSNR分别提升14.61%与4.73%；此外，该模型在巨幅壁画最终修复中取得理想效果。