Infrared and visible image fusion has gradually proved to be a vital fork in the field of multi-modality imaging technologies. In recent developments, researchers not only focus on the quality of fused images but also evaluate their performance in downstream tasks. Nevertheless, the majority of methods seldom put their eyes on the mutual learning from different modalities, resulting in fused images lacking significant details and textures. To overcome this issue, we propose an interactive graph neural network (GNN)-based architecture between cross modality for fusion, called IGNet. Specifically, we first apply a multi-scale extractor to achieve shallow features, which are employed as the necessary input to build graph structures. Then, the graph interaction module can construct the extracted intermediate features of the infrared/visible branch into graph structures. Meanwhile, the graph structures of two branches interact for cross-modality and semantic learning, so that fused images can maintain the important feature expressions and enhance the performance of downstream tasks. Besides, the proposed leader nodes can improve information propagation in the same modality. Finally, we merge all graph features to get the fusion result. Extensive experiments on different datasets (TNO, MFNet and M3FD) demonstrate that our IGNet can generate visually appealing fused images while scoring averagely 2.59% [email protected] and 7.77% mIoU higher in detection and segmentation than the compared state-of-the-art methods. The source code of the proposed IGNet can be available at https://github.com/lok-18/IGNet.

翻译：红外与可见光图像融合已逐渐被证明是多模态成像技术领域的一个重要分支。在近年来的发展中，研究者们不仅关注融合图像的质量，还评估其在下游任务中的性能。然而，大多数方法很少关注不同模态间的相互学习，导致融合图像缺乏显著细节和纹理。为解决这一问题，我们提出了一种基于交互式图神经网络（GNN）的跨模态融合架构，称为IGNet。具体而言，我们首先应用多尺度提取器获得浅层特征，作为构建图结构所需的输入。然后，图交互模块可将红外/可见光分支提取的中间特征构建为图结构。同时，两个分支的图结构进行跨模态和语义学习交互，使融合图像能够保留重要特征表达并增强下游任务性能。此外，我们提出的主导节点可改善同模态内的信息传播。最后，我们融合所有图特征以获得融合结果。在多个数据集（TNO、MFNet和M3FD）上的大量实验表明，我们的IGNet能生成视觉上令人满意的融合图像，同时在检测和分割任务中平均比对比的最新方法高出2.59%的[email protected]和7.77%的mIoU。所提出的IGNet源代码可在https://github.com/lok-18/IGNet获取。