Hyperspectral images (HSI) with abundant spectral information reflected materials property usually perform low spatial resolution due to the hardware limits. Meanwhile, multispectral images (MSI), e.g., RGB images, have a high spatial resolution but deficient spectral signatures. Hyperspectral and multispectral image fusion can be cost-effective and efficient for acquiring both high spatial resolution and high spectral resolution images. Many of the conventional HSI and MSI fusion algorithms rely on known spatial degradation parameters, i.e., point spread function, spectral degradation parameters, spectral response function, or both of them. Another class of deep learning-based models relies on the ground truth of high spatial resolution HSI and needs large amounts of paired training images when working in a supervised manner. Both of these models are limited in practical fusion scenarios. In this paper, we propose an unsupervised HSI and MSI fusion model based on the cycle consistency, called CycFusion. The CycFusion learns the domain transformation between low spatial resolution HSI (LrHSI) and high spatial resolution MSI (HrMSI), and the desired high spatial resolution HSI (HrHSI) are considered to be intermediate feature maps in the transformation networks. The CycFusion can be trained with the objective functions of marginal matching in single transform and cycle consistency in double transforms. Moreover, the estimated PSF and SRF are embedded in the model as the pre-training weights, which further enhances the practicality of our proposed model. Experiments conducted on several datasets show that our proposed model outperforms all compared unsupervised fusion methods. The codes of this paper will be available at this address: https: //github.com/shuaikaishi/CycFusion for reproducibility.

翻译：高光谱图像（HSI）因含有反映物质属性的丰富光谱信息，但由于硬件限制通常空间分辨率较低。而多光谱图像（MSI，如RGB图像）虽具有高空间分辨率，但光谱特征不足。高光谱与多光谱图像融合是一种兼具经济性与高效性的技术，可同时获取高空间分辨率与高光谱分辨率的图像。传统HSI与MSI融合算法大多依赖于已知的空间退化参数（如点扩展函数）、光谱退化参数（如光谱响应函数），或两者兼需。另一类基于深度学习的模型在有监督模式下依赖于高空间分辨率HSI的真实标签，需要大量配对训练图像。这两类模型在实际融合场景中均存在局限。本文提出一种基于循环一致性的无监督HSI与MSI融合模型，称为CycFusion。该模型学习低空间分辨率HSI（LrHSI）与高空间分辨率MSI（HrMSI）之间的域变换，并将所需的高空间分辨率HSI（HrHSI）视为变换网络中的中间特征图。CycFusion可通过单次变换中的边缘匹配目标函数与双重变换中的循环一致性目标函数进行训练。此外，模型嵌入了估计的PSF和SRF作为预训练权重，进一步增强了模型的实用性。在多个数据集上的实验表明，所提模型优于所有对比的无监督融合方法。本文代码将发布于https://github.com/shuaikaishi/CycFusion以支持可复现性。