On a shutter press, modern handheld cameras capture multiple images in rapid succession and merge them to generate a single image. However, individual frames in a burst are misaligned due to inevitable motions and contain multiple degradations. The challenge is to properly align the successive image shots and merge their complimentary information to achieve high-quality outputs. Towards this direction, we propose Burstormer: a novel transformer-based architecture for burst image restoration and enhancement. In comparison to existing works, our approach exploits multi-scale local and non-local features to achieve improved alignment and feature fusion. Our key idea is to enable inter-frame communication in the burst neighborhoods for information aggregation and progressive fusion while modeling the burst-wide context. However, the input burst frames need to be properly aligned before fusing their information. Therefore, we propose an enhanced deformable alignment module for aligning burst features with regards to the reference frame. Unlike existing methods, the proposed alignment module not only aligns burst features but also exchanges feature information and maintains focused communication with the reference frame through the proposed reference-based feature enrichment mechanism, which facilitates handling complex motions. After multi-level alignment and enrichment, we re-emphasize on inter-frame communication within burst using a cyclic burst sampling module. Finally, the inter-frame information is aggregated using the proposed burst feature fusion module followed by progressive upsampling. Our Burstormer outperforms state-of-the-art methods on burst super-resolution, burst denoising and burst low-light enhancement. Our codes and pretrained models are available at https:// github.com/akshaydudhane16/Burstormer

翻译：快门按下时，现代手持相机会快速连续拍摄多张图像并合并生成单张图像。然而，由于不可避免的运动，突发序列中的各帧图像存在错位，且包含多种退化。如何正确对齐连续拍摄的图像帧并融合其互补信息以生成高质量输出，是该领域的关键挑战。为此，我们提出Burstormer：一种基于Transformer架构的新型突发图像恢复与增强模型。与现有工作相比，我们的方法利用多尺度局部与非局部特征实现更优对齐和特征融合。核心思想是通过建模突发图像序列的全局上下文，在突发邻域内实现帧间通信以完成信息聚合与渐进融合。但输入突发帧在融合信息前需先正确对齐，因此我们提出增强型可变形对齐模块，用于将突发特征对齐至参考帧。与现有方法不同，该对齐模块不仅能对齐突发特征，还能通过提出的基于参考的特征增强机制交换特征信息并保持与参考帧的聚焦通信，从而有效处理复杂运动。完成多层级对齐与增强后，我们利用循环突发采样模块重新强化帧间通信。最后，通过提出的突发特征融合模块聚合帧间信息，并进行渐进上采样。我们的Burstormer在突发超分辨率、突发去噪和突发低光增强任务上均优于现有最优方法。代码与预训练模型已开源至https://github.com/akshaydudhane16/Burstormer。