Relying on paired synthetic data, existing learning-based Computational Aberration Correction (CAC) methods are confronted with the intricate and multifaceted synthetic-to-real domain gap, which leads to suboptimal performance in real-world applications. In this paper, in contrast to improving the simulation pipeline, we deliver a novel insight into real-world CAC from the perspective of Unsupervised Domain Adaptation (UDA). By incorporating readily accessible unpaired real-world data into training, we formalize the Domain Adaptive CAC (DACAC) task, and then introduce a comprehensive Real-world aberrated images (Realab) dataset to benchmark it. The setup task presents a formidable challenge due to the intricacy of understanding the target aberration domain. To this intent, we propose a novel Quntized Domain-Mixing Representation (QDMR) framework as a potent solution to the issue. QDMR adapts the CAC model to the target domain from three key aspects: (1) reconstructing aberrated images of both domains by a VQGAN to learn a Domain-Mixing Codebook (DMC) which characterizes the degradation-aware priors; (2) modulating the deep features in CAC model with DMC to transfer the target domain knowledge; and (3) leveraging the trained VQGAN to generate pseudo target aberrated images from the source ones for convincing target domain supervision. Extensive experiments on both synthetic and real-world benchmarks reveal that the models with QDMR consistently surpass the competitive methods in mitigating the synthetic-to-real gap, which produces visually pleasant real-world CAC results with fewer artifacts. Codes and datasets will be made publicly available.
翻译:依托于配对的合成数据,现有的基于学习的计算像差校正(CAC)方法面临着复杂且多层面的合成-真实域差距问题,导致其在真实场景中表现欠佳。本文不同于改进模拟流程,而是从无监督域适应(UDA)的角度为真实场景CAC带来全新见解。通过将易于获取的非配对真实数据融入训练过程,我们正式定义了域自适应CAC(DACAC)任务,并引入包含丰富真实像差图像(Realab)的综合数据集作为基准。由于目标像差域的理解复杂性,该设定任务极具挑战性。为此,我们提出量化域混合表示(QDMR)框架作为该问题的有效解决方案。QDMR从三个方面将CAC模型适应至目标域:(1)利用VQGAN重构双域像差图像,学习刻画退化先验的域混合码本(DMC);(2)通过DMC调制CAC模型深层特征以迁移目标域知识;(3)借助训练完成的VQGAN,从源域图像生成伪目标像差图像,提供可靠的目标域监督。在合成与真实基准上的大量实验表明,采用QDMR的模型在缩小合成-真实域差距方面持续优于竞争方法,能以更少伪影生成视觉愉悦的真实场景CAC结果。代码与数据集将公开发布。