Avoiding the introduction of ghosts when synthesising LDR images as high dynamic range (HDR) images is a challenging task. Convolutional neural networks (CNNs) are effective for HDR ghost removal in general, but are challenging to deal with the LDR images if there are large movements or oversaturation/undersaturation. Existing dual-branch methods combining CNN and Transformer omit part of the information from non-reference images, while the features extracted by the CNN-based branch are bound to the kernel size with small receptive field, which are detrimental to the deblurring and the recovery of oversaturated/undersaturated regions. In this paper, we propose a novel hierarchical dual Transformer method for ghost-free HDR (HDT-HDR) images generation, which extracts global features and local features simultaneously. First, we use a CNN-based head with spatial attention mechanisms to extract features from all the LDR images. Second, the LDR features are delivered to the Hierarchical Dual Transformer (HDT). In each Dual Transformer (DT), the global features are extracted by the window-based Transformer, while the local details are extracted using the channel attention mechanism with deformable CNNs. Finally, the ghost free HDR image is obtained by dimensional mapping on the HDT output. Abundant experiments demonstrate that our HDT-HDR achieves the state-of-the-art performance among existing HDR ghost removal methods.

翻译：在将低动态范围（LDR）图像合成为高动态范围（HDR）图像时，避免鬼影引入是一项具有挑战性的任务。卷积神经网络（CNN）通常能有效去除HDR鬼影，但若LDR图像中存在大幅运动或过曝光/欠曝光现象，则难以处理。现有结合CNN与Transformer的双分支方法会忽略非参考图像中的部分信息，而基于CNN的分支提取的特征受限于卷积核尺寸，感受野较小，不利于去模糊及过曝光/欠曝光区域的恢复。本文提出一种新颖的层次化双Transformer无鬼影HDR图像生成方法（HDT-HDR），该方法可同时提取全局特征与局部特征。首先，使用带空间注意力机制的CNN头部从所有LDR图像中提取特征；其次，将LDR特征输入层次化双Transformer（HDT）。在每个双Transformer（DT）中，基于窗口的Transformer提取全局特征，而采用带可变形CNN的通道注意力机制提取局部细节；最后，通过HDT输出的维度映射获得无鬼影HDR图像。大量实验表明，我们的HDT-HDR方法在现有HDR鬼影去除方法中达到了最优性能。