Images captured under sub-optimal illumination conditions may contain both over- and under-exposures. Current approaches mainly focus on adjusting image brightness, which may exacerbate the color tone distortion in under-exposed areas and fail to restore accurate colors in over-exposed regions. We observe that over- and under-exposed regions display opposite color tone distribution shifts with respect to each other, which may not be easily normalized in joint modeling as they usually do not have ``normal-exposed'' regions/pixels as reference. In this paper, we propose a novel method to enhance images with both over- and under-exposures by learning to estimate and correct such color shifts. Specifically, we first derive the color feature maps of the brightened and darkened versions of the input image via a UNet-based network, followed by a pseudo-normal feature generator to produce pseudo-normal color feature maps. We then propose a novel COlor Shift Estimation (COSE) module to estimate the color shifts between the derived brightened (or darkened) color feature maps and the pseudo-normal color feature maps. The COSE module corrects the estimated color shifts of the over- and under-exposed regions separately. We further propose a novel COlor MOdulation (COMO) module to modulate the separately corrected colors in the over- and under-exposed regions to produce the enhanced image. Comprehensive experiments show that our method outperforms existing approaches. Project webpage: https://github.com/yiyulics/CSEC.

翻译：在非理想光照条件下捕获的图像可能同时包含过曝光和欠曝光区域。现有方法主要侧重于调整图像亮度，这可能会加剧欠曝光区域的色调失真，并且无法准确恢复过曝光区域的真实色彩。我们观察到，过曝光与欠曝光区域呈现出彼此相反的色调分布偏移，由于通常缺乏"正常曝光"区域/像素作为参考，这种偏移在联合建模中难以被归一化。本文提出一种新颖方法，通过学习和校正此类色彩偏移来增强同时存在过曝光与欠曝光的图像。具体而言，我们首先通过基于UNet的网络生成输入图像增亮版与变暗版的色彩特征图，随后利用伪正常特征生成器产生伪正常色彩特征图。继而提出创新的色彩偏移估计模块，用于估计生成的增亮（或变暗）色彩特征图与伪正常色彩特征图之间的色彩偏移。该模块分别校正过曝光与欠曝光区域的估计色彩偏移。进一步提出创新的色彩调制模块，对分别校正后的过曝光与欠曝光区域色彩进行调制，最终生成增强图像。综合实验表明，本方法优于现有技术。项目页面：https://github.com/yiyulics/CSEC。