An adequate fusion of the most significant salient information from multiple input channels is essential for many aerial imaging tasks. While multispectral recordings reveal features in various spectral ranges, synthetic aperture sensing makes occluded features visible. We present a first and hybrid (model- and learning-based) architecture for fusing the most significant features from conventional aerial images with the ones from integral aerial images that are the result of synthetic aperture sensing for removing occlusion. It combines the environment's spatial references with features of unoccluded targets that would normally be hidden by dense vegetation. Our method out-beats state-of-the-art two-channel and multi-channel fusion approaches visually and quantitatively in common metrics, such as mutual information, visual information fidelity, and peak signal-to-noise ratio. The proposed model does not require manually tuned parameters, can be extended to an arbitrary number and combinations of spectral channels, and is reconfigurable for addressing different use cases. We demonstrate examples for search-and-rescue, wildfire detection, and wildlife observation.

翻译：针对多通道输人的显著特征有效融合是多模态航拍成像任务的关键。多光谱成像可揭露不同光谱范围的特征，而合成孔径感知技术则能显现被遮挡的物体。本文首次提出一种混合架构（融合模型驱动与数据驱动方法），用于融合传统航拍图像与通过合成孔径感知消除遮挡获得的积分航拍图像中的显著特征。该架构融合环境空间参考信息与被密植被遮挡的目标特征。在互信息、视觉信息保真度与峰值信噪比等通用指标上，本方法在视觉与定量评估中均超越现有最优的双通道及多通道融合方法。所提模型无需人工参数调整，可扩展至任意数量及组合的光谱通道，并可根据不同应用场景重新配置。我们通过搜索救援、野火检测及野生动物观测等案例验证了其有效性。