Avoiding the introduction of ghosts when synthesising LDR images as high dynamic range (HDR) images is a challenging task. Convolutional neural networks (CNNs) are effective for HDR ghost removal in general, but are challenging to deal with the LDR images if there are large movements or oversaturation/undersaturation. Existing dual-branch methods combining CNN and Transformer omit part of the information from non-reference images, while the features extracted by the CNN-based branch are bound to the kernel size with small receptive field, which are detrimental to the deblurring and the recovery of oversaturated/undersaturated regions. In this paper, we propose a novel hierarchical dual Transformer method for ghost-free HDR (HDT-HDR) images generation, which extracts global features and local features simultaneously. First, we use a CNN-based head with spatial attention mechanisms to extract features from all the LDR images. Second, the LDR features are delivered to the Hierarchical Dual Transformer (HDT). In each Dual Transformer (DT), the global features are extracted by the window-based Transformer, while the local details are extracted using the channel attention mechanism with deformable CNNs. Finally, the ghost free HDR image is obtained by dimensional mapping on the HDT output. Abundant experiments demonstrate that our HDT-HDR achieves the state-of-the-art performance among existing HDR ghost removal methods.

翻译：在将低动态范围（LDR）图像合成为高动态范围（HDR）图像时，避免鬼影的引入是一项极具挑战性的任务。卷积神经网络（CNNs）通常能有效去除HDR鬼影，但当LDR图像存在大幅运动或过曝光/欠曝光时，CNNs的处理变得困难。现有结合CNN与Transformer的双分支方法会忽略非参考图像的部分信息，而基于CNN的分支所提取的特征受限于小感受野的核尺寸，这不利于去模糊以及过曝光/欠曝光区域的恢复。本文提出一种新颖的层次化双Transformer方法，用于生成无鬼影HDR图像（HDT-HDR），该方法能同时提取全局特征与局部特征。首先，我们采用基于CNN的头部结构结合空间注意力机制，从所有LDR图像中提取特征。其次，将LDR特征输入层次化双Transformer（HDT）。在每个双Transformer（DT）中，全局特征由基于窗口的Transformer提取，而局部细节则通过可变形CNN的通道注意力机制提取。最后，通过对HDT输出进行维度映射得到无鬼影HDR图像。大量实验表明，所提出的HDT-HDR在现有HDR鬼影去除方法中达到了最优性能。