RAW to sRGB mapping, which aims to convert RAW images from smartphones into RGB form equivalent to that of Digital Single-Lens Reflex (DSLR) cameras, has become an important area of research. However, current methods often ignore the difference between cell phone RAW images and DSLR camera RGB images, a difference that goes beyond the color matrix and extends to spatial structure due to resolution variations. Recent methods directly rebuild color mapping and spatial structure via shared deep representation, limiting optimal performance. Inspired by Image Signal Processing (ISP) pipeline, which distinguishes image restoration and enhancement, we present a novel Neural ISP framework, named FourierISP. This approach breaks the image down into style and structure within the frequency domain, allowing for independent optimization. FourierISP is comprised of three subnetworks: Phase Enhance Subnet for structural refinement, Amplitude Refine Subnet for color learning, and Color Adaptation Subnet for blending them in a smooth manner. This approach sharpens both color and structure, and extensive evaluations across varied datasets confirm that our approach realizes state-of-the-art results. Code will be available at ~\url{https://github.com/alexhe101/FourierISP}.
翻译:RAW到sRGB映射旨在将智能手机拍摄的RAW图像转换为与数码单反相机(DSLR)等效的RGB格式,已成为研究的重要领域。然而,当前方法常忽略手机RAW图像与DSLR相机RGB图像间的差异——这种差异不仅体现在颜色矩阵上,更因分辨率差异延伸至空间结构。近期方法试图通过共享深度表征直接重建颜色映射与空间结构,限制了最优性能。受图像信号处理(ISP)流水线区分图像恢复与增强的启发,我们提出一种新型神经ISP框架——FourierISP。该方法将图像在频域内分解为风格与结构,实现独立优化。FourierISP由三个子网络构成:用于结构细化的相位增强子网络、用于颜色学习的幅度精化子网络,以及以平滑方式融合二者的颜色适应子网络。此方法显著锐化了色彩与结构,跨多样化数据集的广泛评估证实了该方法实现了最先进的性能。代码将发布于~\url{https://github.com/alexhe101/FourierISP}。