Most existing learning-based infrared and visible image fusion (IVIF) methods exhibit massive redundant information in the fusion images, i.e., yielding edge-blurring effect or unrecognizable for object detectors. To alleviate these issues, we propose a semantic structure-preserving approach for IVIF, namely SSPFusion. At first, we design a Structural Feature Extractor (SFE) to extract the structural features of infrared and visible images. Then, we introduce a multi-scale Structure-Preserving Fusion (SPF) module to fuse the structural features of infrared and visible images, while maintaining the consistency of semantic structures between the fusion and source images. Owing to these two effective modules, our method is able to generate high-quality fusion images from pairs of infrared and visible images, which can boost the performance of downstream computer-vision tasks. Experimental results on three benchmarks demonstrate that our method outperforms eight state-of-the-art image fusion methods in terms of both qualitative and quantitative evaluations. The code for our method, along with additional comparison results, will be made available at: https://github.com/QiaoYang-CV/SSPFUSION.
翻译:现有大多数基于学习的红外与可见光图像融合(IVIF)方法在融合图像中呈现大量冗余信息,例如产生边缘模糊效应或导致目标检测器无法识别。为缓解这些问题,我们提出一种面向IVIF的语义结构保持方法,即SSPFusion。首先,我们设计了一个结构特征提取器(SFE)用于提取红外与可见光图像的结构特征。其次,引入多尺度结构保持融合(SPF)模块,在融合红外与可见光图像结构特征的同时,保持融合图像与源图像之间语义结构的一致性。得益于这两个有效模块,我们的方法能够从红外与可见光图像对中生成高质量的融合图像,从而提升下游计算机视觉任务的性能。在三个基准数据集上的实验结果表明,无论在定性还是定量评估方面,我们的方法均优于八种当前最先进的图像融合方法。本方法的代码及更多对比结果将发布在:https://github.com/QiaoYang-CV/SSPFUSION。