In this paper, we introduce a new approach for high-quality multi-exposure image fusion (MEF). We show that the fusion weights of an exposure can be encoded into a 1D lookup table (LUT), which takes pixel intensity value as input and produces fusion weight as output. We learn one 1D LUT for each exposure, then all the pixels from different exposures can query 1D LUT of that exposure independently for high-quality and efficient fusion. Specifically, to learn these 1D LUTs, we involve attention mechanism in various dimensions including frame, channel and spatial ones into the MEF task so as to bring us significant quality improvement over the state-of-the-art (SOTA). In addition, we collect a new MEF dataset consisting of 960 samples, 155 of which are manually tuned by professionals as ground-truth for evaluation. Our network is trained by this dataset in an unsupervised manner. Extensive experiments are conducted to demonstrate the effectiveness of all the newly proposed components, and results show that our approach outperforms the SOTA in our and another representative dataset SICE, both qualitatively and quantitatively. Moreover, our 1D LUT approach takes less than 4ms to run a 4K image on a PC GPU. Given its high quality, efficiency and robustness, our method has been shipped into millions of Android mobiles across multiple brands world-wide. Code is available at: https://github.com/Hedlen/MEFLUT.
翻译:本文提出了一种高质量多曝光图像融合(MEF)的新方法。我们证明,曝光图像的融合权重可编码为一维查找表(LUT),该表以像素强度值为输入,输出融合权重。为每次曝光学习一个一维LUT后,不同曝光下的所有像素可独立查询该曝光对应的一维LUT,从而实现高质量且高效的融合。具体而言,为学习这些一维LUT,我们将不同维度的注意力机制(包括帧维、通道维和空间维)引入MEF任务,从而显著提升融合质量,超越现有最优方法(SOTA)。此外,我们构建了一个包含960个样本的新MEF数据集,其中155个样本由专业人员手动调优作为真值用于评估。网络在该数据集上以无监督方式训练。通过大量实验验证了新提出各组件的有效性,结果表明,在本数据集及另一个代表性数据集SICE上,我们的方法在定性和定量指标上均优于SOTA。此外,采用一维LUT方法在PC GPU上处理4K图像仅需不到4毫秒。凭借其高质量、高效性和鲁棒性,该方法已推广至全球数百万台跨品牌安卓手机。代码开源地址:https://github.com/Hedlen/MEFLUT。