Low dynamic range (LDR) cameras cannot deal with wide dynamic range inputs, frequently leading to local overexposure issues. We present a learning-based system to reduce these artifacts without resorting to complex acquisition mechanisms like alternating exposures or costly processing that are typical of high dynamic range (HDR) imaging. We propose a transformer-based deep neural network (DNN) to infer the missing HDR details. In an ablation study, we show the importance of using a multiscale DNN and train it with the proper cost function to achieve state-of-the-art quality. To aid the reconstruction of the overexposed areas, our DNN takes a reference frame from the past as an additional input. This leverages the commonly occurring temporal instabilities of autoexposure to our advantage: since well-exposed details in the current frame may be overexposed in the future, we use reinforcement learning to train a reference frame selection DNN that decides whether to adopt the current frame as a future reference. Without resorting to alternating exposures, we obtain therefore a causal, HDR hallucination algorithm with potential application in common video acquisition settings. Our demo video can be found at https://drive.google.com/file/d/1-r12BKImLOYCLUoPzdebnMyNjJ4Rk360/view
翻译:低动态范围(LDR)相机无法处理宽动态范围输入,常导致局部过曝问题。我们提出一种基于学习的系统,在不采用高动态范围(HDR)成像中典型的交替曝光或昂贵处理等复杂采集机制的情况下,减少此类伪影。我们提出基于Transformer的深度神经网络(DNN)来推断缺失的HDR细节。消融研究表明,采用多尺度DNN并配合适当代价函数进行训练,可获得最先进的质量。为辅助过曝区域重建,我们的DNN额外引入历史参考帧,利用自动曝光常见的时间不稳定性:当前帧曝光良好的细节在未来可能过曝,因此我们采用强化学习训练参考帧选择DNN,决定是否将当前帧作为未来参考。在不依赖交替曝光的情况下,我们获得了一种因果性的HDR修复算法,可应用于常规视频采集场景。演示视频见https://drive.google.com/file/d/1-r12BKImLOYCLUoPzdebnMyNjJ4Rk360/view