RAW images are unprocessed camera sensor output with sensor-specific RGB values based on the sensor's color filter spectral sensitivities. RAW images also incur strong color casts due to the sensor's response to the spectral properties of scene illumination. The sensor- and illumination-specific nature of RAW images makes it challenging to capture RAW datasets for deep learning methods, as scenes need to be captured for each sensor and under a wide range of illumination. Methods for illumination augmentation for a given sensor and the ability to map RAW images between sensors are important for reducing the burden of data capture. To explore this problem, we introduce the first-of-its-kind dataset comprising carefully captured scenes under a wide range of illumination. Specifically, we use a customized lightbox with tunable illumination spectra to capture several scenes with different cameras. Our illumination and sensor mapping dataset has 390 illuminations, four cameras, and 18 scenes. Using this dataset, we introduce a lightweight neural network approach for illumination and sensor mapping that outperforms competing methods. We demonstrate the utility of our approach on the downstream task of training a neural ISP. Link to project page: https://github.com/SamsungLabs/illum-sensor-mapping.

翻译：RAW图像是未经处理的相机传感器输出，其传感器特定的RGB值基于传感器的彩色滤光片光谱灵敏度。由于传感器对场景光照光谱特性的响应，RAW图像还会产生强烈的色偏。RAW图像的传感器特定性和光照特定性使得为深度学习方法捕获RAW数据集具有挑战性，因为需要针对每个传感器并在各种光照条件下捕获场景。针对给定传感器的光照增强方法以及在不同传感器间映射RAW图像的能力，对于减轻数据捕获负担至关重要。为探索此问题，我们引入了首个包含在各种光照条件下精心捕获场景的数据集。具体而言，我们使用具有可调光谱的自定义灯箱，配合多款不同相机捕获了多个场景。我们的光照与传感器映射数据集包含390种光照、四款相机和18个场景。利用此数据集，我们提出了一种轻量级神经网络方法用于光照与传感器映射，其性能优于现有方法。我们在训练神经图像信号处理器的下游任务中展示了所提方法的实用性。项目页面链接：https://github.com/SamsungLabs/illum-sensor-mapping。