As the physical size of recent CMOS image sensors (CIS) gets smaller, the latest mobile cameras are adopting unique non-Bayer color filter array (CFA) patterns (e.g., Quad, Nona, QxQ), which consist of homogeneous color units with adjacent pixels. These non-Bayer sensors are superior to conventional Bayer CFA thanks to their changeable pixel-bin sizes for different light conditions but may introduce visual artifacts during demosaicing due to their inherent pixel pattern structures and sensor hardware characteristics. Previous demosaicing methods have primarily focused on Bayer CFA, necessitating distinct reconstruction methods for non-Bayer patterned CIS with various CFA modes under different lighting conditions. In this work, we propose an efficient unified demosaicing method that can be applied to both conventional Bayer RAW and various non-Bayer CFAs' RAW data in different operation modes. Our Knowledge Learning-based demosaicing model for Adaptive Patterns, namely KLAP, utilizes CFA-adaptive filters for only 1% key filters in the network for each CFA, but still manages to effectively demosaic all the CFAs, yielding comparable performance to the large-scale models. Furthermore, by employing meta-learning during inference (KLAP-M), our model is able to eliminate unknown sensor-generic artifacts in real RAW data, effectively bridging the gap between synthetic images and real sensor RAW. Our KLAP and KLAP-M methods achieved state-of-the-art demosaicing performance in both synthetic and real RAW data of Bayer and non-Bayer CFAs.

翻译：随着近年来CMOS图像传感器（CIS）物理尺寸不断缩小，最新移动相机纷纷采用由同色像素单元组成的非拜耳色彩滤波阵列（CFA）模式（如Quad、Nona、QxQ）。相较于传统拜耳CFA，非拜耳传感器因其可随光照条件调整像素合并尺寸而具有优势，但其固有的像素模式结构与传感器硬件特性会在去马赛克过程中引入视觉伪影。现有去马赛克方法主要针对拜耳CFA设计，导致在不同光照条件下需为不同CFA模式的非拜耳CIS开发独立的重建方法。本文提出一种高效统一的去马赛克方法，可同时处理传统拜耳RAW数据与多种非拜耳CFA在不同工作模式下的RAW数据。我们所提出的自适应模式知识学习去马赛克模型（简称KLAP），通过网络中每个CFA仅占1%的关键滤波器实现CFA自适应滤波，却能有效完成所有CFA的去马赛克任务，性能可比肩大规模模型。进一步地，通过在推理阶段引入元学习（KLAP-M），本模型可消除真实RAW数据中未知的传感器通用伪影，有效弥合合成图像与真实传感器RAW之间的差距。我们的KLAP与KLAP-M方法在处理拜耳及非拜耳CFA的合成与真实RAW数据时，均取得了最先进的去马赛克性能。