Image fusion typically employs non-invertible neural networks to merge multiple source images into a single fused image. However, for clinical experts, solely relying on fused images may be insufficient for making diagnostic decisions, as the fusion mechanism blends features from source images, thereby making it difficult to interpret the underlying tumor pathology. We introduce FusionINN, a novel invertible image fusion framework, capable of efficiently generating fused images and also decomposing them back to the source images by solving the inverse of the fusion process. FusionINN guarantees lossless one-to-one pixel mapping by integrating a normally distributed latent image alongside the fused image to facilitate the generative modeling of the decomposition process. To the best of our knowledge, we are the first to investigate the decomposability of fused images, which is particularly crucial for life-sensitive applications such as medical image fusion compared to other tasks like multi-focus or multi-exposure image fusion. Our extensive experimentation validates FusionINN over existing discriminative and generative fusion methods, both subjectively and objectively. Moreover, compared to a recent denoising diffusion-based fusion model, our approach offers faster and qualitatively better fusion results. We also exhibit the clinical utility of our results in aiding disease prognosis.
翻译:图像融合通常采用不可逆神经网络将多源图像合并为单一融合图像。然而,对于临床专家而言,仅依赖融合图像可能不足以做出诊断决策,因为融合机制混合了源图像的特征,导致难以解读潜在的肿瘤病理。我们提出FusionINN,一种新颖的可逆图像融合框架,能够高效生成融合图像,并通过求解融合过程的逆变换将其分解回源图像。FusionINN通过整合正态分布的潜变量图像与融合图像,确保了无损的一对一像素映射,从而促进了分解过程的生成式建模。据我们所知,我们是首个研究融合图像可分解性的团队,与多聚焦或多曝光融合等其他任务相比,这一特性对医疗图像融合等生命敏感型应用尤为关键。广泛的实验在主观和客观评估上验证了FusionINN优于现有的判别式和生成式融合方法。此外,与最近基于去噪扩散的融合模型相比,我们的方法能更快地生成更高质量的融合结果。我们还展示了结果在辅助疾病预后中的临床实用性。