Infrared and visible image fusion (IVIF) is a crucial technique for enhancing visual performance by integrating unique information from different modalities into one fused image. Exiting methods pay more attention to conducting fusion with undisturbed data, while overlooking the impact of deliberate interference on the effectiveness of fusion results. To investigate the robustness of fusion models, in this paper, we propose a novel adversarial attack resilient network, called $\textrm{A}^{\textrm{2}}$RNet. Specifically, we develop an adversarial paradigm with an anti-attack loss function to implement adversarial attacks and training. It is constructed based on the intrinsic nature of IVIF and provide a robust foundation for future research advancements. We adopt a Unet as the pipeline with a transformer-based defensive refinement module (DRM) under this paradigm, which guarantees fused image quality in a robust coarse-to-fine manner. Compared to previous works, our method mitigates the adverse effects of adversarial perturbations, consistently maintaining high-fidelity fusion results. Furthermore, the performance of downstream tasks can also be well maintained under adversarial attacks. Code is available at https://github.com/lok-18/A2RNet.
翻译:红外与可见光图像融合(IVIF)是一种通过整合不同模态的独特信息到单幅融合图像中,以增强视觉性能的关键技术。现有方法更多关注在无干扰数据上进行融合,而忽视了故意干扰对融合结果有效性的影响。为探究融合模型的鲁棒性,本文提出了一种新型抗对抗攻击网络,称为A²RNet。具体而言,我们开发了一种包含抗攻击损失函数的对抗范式,以实现对抗攻击与训练。该范式基于IVIF的内在特性构建,为未来研究进展提供了鲁棒基础。在此范式下,我们采用Unet作为主干网络,并结合基于Transformer的防御性细化模块(DRM),以鲁棒的由粗到细方式保证融合图像质量。与先前工作相比,我们的方法减轻了对抗性扰动的不利影响,始终保持高保真度的融合结果。此外,下游任务的性能在对抗攻击下也能得到良好保持。代码发布于https://github.com/lok-18/A2RNet。