Modern smartphone camera quality heavily relies on the image signal processor (ISP) to enhance captured raw images, utilizing carefully designed modules to produce final output images encoded in a standard color space (e.g., sRGB). Neural-based end-to-end learnable ISPs offer promising advancements, potentially replacing traditional ISPs with their ability to adapt without requiring extensive tuning for each new camera model, as is often the case for nearly every module in traditional ISPs. However, the key challenge with the recent learning-based ISPs is the urge to collect large paired datasets for each distinct camera model due to the influence of intrinsic camera characteristics on the formation of input raw images. This paper tackles this challenge by introducing a novel method for unpaired learning of raw-to-raw translation across diverse cameras. Specifically, we propose Rawformer, an unsupervised Transformer-based encoder-decoder method for raw-to-raw translation. It accurately maps raw images captured by a certain camera to the target camera, facilitating the generalization of learnable ISPs to new unseen cameras. Our method demonstrates superior performance on real camera datasets, achieving higher accuracy compared to previous state-of-the-art techniques, and preserving a more robust correlation between the original and translated raw images. The codes and the pretrained models are available at https://github.com/gosha20777/rawformer.

翻译：现代智能手机的成像质量在很大程度上依赖于图像信号处理器（ISP）来增强捕获的原始图像，其通过精心设计的模块生成标准色彩空间（如sRGB）编码的最终输出图像。基于神经网络的端到端可学习ISP展现出显著优势，有望替代传统ISP——传统ISP的几乎所有模块通常都需要针对每个新相机型号进行大量调优，而可学习ISP具备自适应能力，无需此类繁琐调整。然而，当前基于学习的ISP面临的核心挑战在于：由于相机固有特性对输入原始图像形成过程的影响，需要为每个不同的相机型号收集大量配对数据集。本文通过提出一种跨相机无配对原始图像到原始图像转换的新方法来解决这一挑战。具体而言，我们提出Rawformer，一种基于Transformer的无监督编码器-解码器方法，用于实现原始图像到原始图像的转换。该方法能够准确地将特定相机捕获的原始图像映射到目标相机域，从而促进可学习ISP对未知新相机的泛化能力。我们的方法在真实相机数据集上表现出卓越性能，相比现有先进技术实现了更高的精度，并在原始图像与转换后图像之间保持了更稳健的关联性。代码与预训练模型已公开于https://github.com/gosha20777/rawformer。