Current AI-based methods do not provide comprehensible physical interpretations of the utilized data, extracted features, and predictions/inference operations. As a result, deep learning models trained using high-resolution satellite imagery lack transparency and explainability and can be merely seen as a black box, which limits their wide-level adoption. Experts need help understanding the complex behavior of AI models and the underlying decision-making process. The explainable artificial intelligence (XAI) field is an emerging field providing means for robust, practical, and trustworthy deployment of AI models. Several XAI techniques have been proposed for image classification tasks, whereas the interpretation of image segmentation remains largely unexplored. This paper offers to bridge this gap by adapting the recent XAI classification algorithms and making them usable for muti-class image segmentation, where we mainly focus on buildings' segmentation from high-resolution satellite images. To benchmark and compare the performance of the proposed approaches, we introduce a new XAI evaluation methodology and metric based on "Entropy" to measure the model uncertainty. Conventional XAI evaluation methods rely mainly on feeding area-of-interest regions from the image back to the pre-trained (utility) model and then calculating the average change in the probability of the target class. Those evaluation metrics lack the needed robustness, and we show that using Entropy to monitor the model uncertainty in segmenting the pixels within the target class is more suitable. We hope this work will pave the way for additional XAI research for image segmentation and applications in the remote sensing discipline.

翻译：当前基于人工智能的方法无法对所用数据、提取特征及预测/推理操作提供可理解的物理解释。因此，利用高分辨率卫星影像训练的深度学习模型缺乏透明度和可解释性，仅能被视为黑箱系统，这限制了其广泛应用。专家需要理解AI模型复杂行为及底层决策过程的帮助。可解释人工智能（XAI）领域作为新兴领域，为AI模型的稳健、实用及可信部署提供了手段。目前已有多项XAI技术针对图像分类任务提出，但图像分割的解释性仍鲜有探索。本文通过适配最新的XAI分类算法并使其适用于多类别图像分割，致力于弥合这一空白——主要聚焦于高分辨率卫星影像中的建筑物分割。为基准测试和比较所提出方法的性能，我们引入了一种基于"熵"的XAI评估新方法论与指标，用于度量模型不确定性。传统XAI评估方法主要依赖将图像中感兴趣区域反馈至预训练（效用）模型，而后计算目标类别概率的平均变化。此类评估指标缺乏所需稳健性，而我们证明利用熵监控模型在分割目标类别像素时的不确定性更为适宜。期望此项工作能为遥感学科中图像分割相关的XAI研究开辟新路径。