Infrared and visible image fusion aim to integrate modality strengths for visually enhanced, informative images. Visible imaging in real-world scenarios is susceptible to dynamic environmental brightness fluctuations, leading to texture degradation. Existing fusion methods lack robustness against such brightness perturbations, significantly compromising the visual fidelity of the fused imagery. To address this challenge, we propose the Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion), which achieves robust image fusion despite dynamic brightness fluctuations. Specifically, we introduce a Brightness Adaptive Gate (BAG) module, which is designed to dynamically select features from brightness-related channels for normalization, while preserving brightness-independent structural information within the source images. Furthermore, we propose a brightness consistency loss function to optimize the BAG module. The entire framework is tuned via alternating training strategies. Extensive experiments validate that our method surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels. Our code is available: https://github.com/SunYM2020/BA-Fusion.
翻译:红外与可见光图像融合旨在整合不同模态的优势,以生成视觉增强且信息丰富的图像。现实场景中的可见光成像易受动态环境亮度波动的影响,导致纹理退化。现有融合方法对此类亮度扰动缺乏鲁棒性,显著降低了融合图像的视觉保真度。为应对这一挑战,我们提出亮度自适应多模态动态融合框架(BA-Fusion),该框架能够在动态亮度波动下实现鲁棒的图像融合。具体而言,我们设计了亮度自适应门控(BAG)模块,该模块通过动态选择亮度相关通道特征进行归一化处理,同时保留源图像中与亮度无关的结构信息。此外,我们提出了亮度一致性损失函数以优化BAG模块。整个框架通过交替训练策略进行调优。大量实验表明,本方法在保留多模态图像信息与视觉保真度方面优于现有先进方法,并在不同亮度条件下表现出卓越的鲁棒性。代码已开源:https://github.com/SunYM2020/BA-Fusion。