Deep learning based fusion methods have been achieving promising performance in image fusion tasks. This is attributed to the network architecture that plays a very important role in the fusion process. However, in general, it is hard to specify a good fusion architecture, and consequently, the design of fusion networks is still a black art, rather than science. To address this problem, we formulate the fusion task mathematically, and establish a connection between its optimal solution and the network architecture that can implement it. This approach leads to a novel method proposed in the paper of constructing a lightweight fusion network. It avoids the time-consuming empirical network design by a trial-and-test strategy. In particular we adopt a learnable representation approach to the fusion task, in which the construction of the fusion network architecture is guided by the optimisation algorithm producing the learnable model. The low-rank representation (LRR) objective is the foundation of our learnable model. The matrix multiplications, which are at the heart of the solution are transformed into convolutional operations, and the iterative process of optimisation is replaced by a special feed-forward network. Based on this novel network architecture, an end-to-end lightweight fusion network is constructed to fuse infrared and visible light images. Its successful training is facilitated by a detail-to-semantic information loss function proposed to preserve the image details and to enhance the salient features of the source images. Our experiments show that the proposed fusion network exhibits better fusion performance than the state-of-the-art fusion methods on public datasets. Interestingly, our network requires a fewer training parameters than other existing methods.

翻译：基于深度学习的融合方法在图像融合任务中已取得显著成效，这归功于网络架构在融合过程中发挥的关键作用。然而，一般而言，设计出优秀的融合架构极为困难，因此融合网络的设计仍是一门"玄学"而非科学。为解决此问题，我们对融合任务进行数学建模，建立其最优解与可实现该解的网络架构之间的关联。该方法引出了本文提出的新型轻量级融合网络构建策略，避免了通过试错法进行耗时的经验性网络设计。具体而言，我们采用可学习表示方法处理融合任务，通过生成可学习模型的优化算法来指导融合网络架构的构建。低秩表示（LRR）目标是构建可学习模型的基础，我们将矩阵乘法（该解法的核心操作）转化为卷积运算，并用特殊的前馈网络替代迭代优化过程。基于这种新型网络架构，我们构建了端到端的轻量级融合网络用于红外与可见光图像融合。所提出的细节-语义信息损失函数有效促进了模型训练，该损失函数旨在保留图像细节并增强源图像的显著特征。实验表明，在公开数据集上，所提融合网络展现出优于当前最优融合方法的性能。值得注意的是，与其他现有方法相比，我们的网络需要更少的训练参数。