Infrared and visible image fusion (IVIF) aims to extract and integrate the complementary information in two different modalities to generate high-quality fused images with salient targets and abundant texture details. However, current image fusion methods go to great lengths to excavate complementary features, which is generally achieved through two efforts. On the one hand, the feature extraction network is expected to have excellent performance in extracting complementary information. On the other hand, complex fusion strategies are often designed to aggregate the complementary information. In other words, enabling the network to perceive and extract complementary information is extremely challenging. Complicated fusion strategies, while effective, still run the risk of losing weak edge details. To this end, this paper rethinks the IVIF outside the box, proposing a complementary-redundant information transfer network (C-RITNet). It reasonably transfers complementary information into redundant one, which integrates both the shared and complementary features from two modalities. Hence, the proposed method is able to alleviate the challenges posed by the complementary information extraction and reduce the reliance on sophisticated fusion strategies. Specifically, to skillfully sidestep aggregating complementary information in IVIF, we first design the mutual information transfer (MIT) module to mutually represent features from two modalities, roughly transferring complementary information into redundant one. Then, a redundant information acquisition supervised by source image (RIASSI) module is devised to further ensure the complementary-redundant information transfer after MIT. Meanwhile, we also propose a structure information preservation (SIP) module to guarantee that the edge structure information of the source images can be transferred to the fusion results.
翻译:红外与可见光图像融合(IVIF)旨在提取并整合两种不同模态的互补信息,以生成具有显著目标和丰富纹理细节的高质量融合图像。然而,当前图像融合方法为挖掘互补特征投入了大量努力,这通常通过两种途径实现:一方面,期望特征提取网络在提取互补信息方面具有卓越性能;另一方面,常设计复杂的融合策略来聚合互补信息。换言之,使网络感知并提取互补信息极具挑战性。复杂的融合策略虽然有效,但仍存在丢失微弱边缘细节的风险。为此,本文打破常规重新思考IVIF,提出一种互补-冗余信息传输网络(C-RITNet),其将互补信息合理转化为冗余信息,从而融合两种模态的共享特征与互补特征。因此,所提方法能够缓解互补信息提取带来的挑战,并减少对复杂融合策略的依赖。具体而言,为巧妙避开IVIF中互补信息的聚合,我们首先设计互信息传输(MIT)模块,使两种模态的特征相互表征,将互补信息初步转化为冗余信息;随后,设计源图像监督的冗余信息获取(RIASSI)模块,进一步确保MIT后的互补-冗余信息传输;同时,提出结构信息保留(SIP)模块,保证源图像的边缘结构信息能够迁移至融合结果中。