In digital images, the performance of optical aberration is a multivariate degradation, where the spectral of the scene, the lens imperfections, and the field of view together contribute to the results. Besides eliminating it at the hardware level, the post-processing system, which utilizes various prior information, is significant for correction. However, due to the content differences among priors, the pipeline that aligns these factors shows limited efficiency and unoptimized restoration. Here, we propose a prior quantization model to correct the optical aberrations in image processing systems. To integrate these messages, we encode various priors into a latent space and quantify them by the learnable codebooks. After quantization, the prior codes are fused with the image restoration branch to realize targeted optical aberration correction. Comprehensive experiments demonstrate the flexibility of the proposed method and validate its potential to accomplish targeted restoration for a specific camera. Furthermore, our model promises to analyze the correlation between the various priors and the optical aberration of devices, which is helpful for joint soft-hardware design.
翻译:在数字图像中,光学像差的表现是一种多变量退化过程,场景的光谱信息、镜头缺陷以及视场角共同影响了最终结果。除了在硬件层面消除像差外,利用多种先验信息的后处理系统对校正确有重要意义。然而,由于不同先验之间存在内容差异,对齐这些因素的处理流程效率有限且复原效果未达最优。本文提出一种先验量化模型,用于校正图像处理系统中的光学像差。为整合这些信息,我们将多种先验编码到一个潜在空间中,并通过可学习的码本对其进行量化。量化后的先验码与图像复原分支相融合,实现针对性的光学像差校正。大量实验证明了所提方法的灵活性,并验证其在特定相机上实现靶向复原的潜力。此外,该模型有望分析不同先验与设备光学像差之间的相关性,这有助于软硬件联合设计。