Infrared and visible image fusion (IVIF) aims to extract and integrate the complementary information in two different modalities to generate high-quality fused images with salient targets and abundant texture details. However, current image fusion methods go to great lengths to excavate complementary features, which is generally achieved through two efforts. On the one hand, the feature extraction network is expected to have excellent performance in extracting complementary information. On the other hand, complex fusion strategies are often designed to aggregate the complementary information. In other words, enabling the network to perceive and extract complementary information is extremely challenging. Complicated fusion strategies, while effective, still run the risk of losing weak edge details. To this end, this paper rethinks the IVIF outside the box, proposing a complementary-redundant information transfer network (C-RITNet). It reasonably transfers complementary information into redundant one, which integrates both the shared and complementary features from two modalities. Hence, the proposed method is able to alleviate the challenges posed by the complementary information extraction and reduce the reliance on sophisticated fusion strategies. Specifically, to skillfully sidestep aggregating complementary information in IVIF, we first design the mutual information transfer (MIT) module to mutually represent features from two modalities, roughly transferring complementary information into redundant one. Then, a redundant information acquisition supervised by source image (RIASSI) module is devised to further ensure the complementary-redundant information transfer after MIT. Meanwhile, we also propose a structure information preservation (SIP) module to guarantee that the edge structure information of the source images can be transferred to the fusion results.
翻译:红外与可见光图像融合旨在提取并整合两种不同模态的互补信息,以生成具有显著目标和丰富纹理细节的高质量融合图像。然而,当前图像融合方法为挖掘互补特征不遗余力,通常通过两方面实现:一方面,期望特征提取网络具备卓越的互补信息提取能力;另一方面,常设计复杂的融合策略以聚合互补信息。换言之,使网络感知并提取互补信息极具挑战性。复杂的融合策略虽有效,但仍存在丢失微弱边缘细节的风险。为此,本文跳出固有框架重新思考IVIF,提出一种互补-冗余信息传输网络(C-RITNet)。该网络将互补信息合理转化为冗余信息,从而整合来自两种模态的共享特征与互补特征。因此,所提方法能缓解互补信息提取带来的挑战,并降低对复杂融合策略的依赖。具体而言,为巧妙规避IVIF中互补信息的聚合,本文首先设计互信息传输(MIT)模块,通过两种模态特征的相互表征,将互补信息初步转化为冗余信息;随后,设计源图像监督的冗余信息获取(RIASSI)模块,进一步确保MIT之后的互补-冗余信息传输;同时,提出结构信息保持(SIP)模块,以保证源图像的边缘结构信息可传递至融合结果。