Numerous ideas have emerged for designing fusion rules in the image fusion field. Essentially, all the existing formulations try to manage the diverse levels of information communicated by the source images to achieve the best fusion result. We argue that there is a scope for improving the performance of existing methods further with the help of FusionBooster, a fusion guidance method proposed in this paper. Our booster is based on the divide and conquer strategy controlled by an information probe. The booster is composed of three building blocks: the probe units, the booster layer, and the assembling module. Given the embedding produced by a backbone method, the probe units assess the source images and divide them according to their information content. This is instrumental in identifying missing information, as a step to its recovery. The recovery of the degraded components along with the fusion guidance are embedded in the booster layer. Lastly, the assembling module is responsible for piecing these advanced components together to deliver the output. We use concise reconstruction loss functions and lightweight models to formulate the network, with marginal computational increase. The experimental results obtained in various fusion tasks, as well as downstream detection tasks, consistently demonstrate that the proposed FusionBooster significantly improves the performance. Our codes will be publicly available on the project homepage.
翻译:在图像融合领域,已有众多针对融合规则设计的思想被提出。本质上,现有方法均尝试管理源图像所传递的不同信息层级,以期获得最佳融合结果。我们认为,借助本文提出的融合引导方法FusionBooster,现有方法的性能仍有进一步提升空间。该增强器基于由信息探测器控制的分治策略。增强器由三个构建模块组成:探测器单元、增强层和组装模块。给定骨干方法生成的嵌入表示后,探测器单元评估源图像并根据其信息含量对其进行划分。这有助于识别缺失信息,作为恢复缺失信息的步骤。退化分量的恢复与融合引导共同嵌入增强层中。最后,组装模块负责将这些高级分量整合以生成输出。我们采用简洁的重构损失函数和轻量级模型构建网络,仅带来边际计算开销。在多种融合任务以及下游检测任务上获得的实验结果一致表明,所提出的FusionBooster显著提升了性能。我们的代码将在项目主页上公开。