Current AI-based methods do not provide comprehensible physical interpretations of the utilized data, extracted features, and predictions/inference operations. As a result, deep learning models trained using high-resolution satellite imagery lack transparency and explainability and can be merely seen as a black box, which limits their wide-level adoption. Experts need help understanding the complex behavior of AI models and the underlying decision-making process. The explainable artificial intelligence (XAI) field is an emerging field providing means for robust, practical, and trustworthy deployment of AI models. Several XAI techniques have been proposed for image classification tasks, whereas the interpretation of image segmentation remains largely unexplored. This paper offers to bridge this gap by adapting the recent XAI classification algorithms and making them usable for muti-class image segmentation, where we mainly focus on buildings' segmentation from high-resolution satellite images. To benchmark and compare the performance of the proposed approaches, we introduce a new XAI evaluation methodology and metric based on "Entropy" to measure the model uncertainty. Conventional XAI evaluation methods rely mainly on feeding area-of-interest regions from the image back to the pre-trained (utility) model and then calculating the average change in the probability of the target class. Those evaluation metrics lack the needed robustness, and we show that using Entropy to monitor the model uncertainty in segmenting the pixels within the target class is more suitable. We hope this work will pave the way for additional XAI research for image segmentation and applications in the remote sensing discipline.

翻译：当前基于人工智能的方法无法对所使用数据、提取特征及预测/推理操作提供可理解的物理解释。因此，基于高分辨率卫星图像训练的深度学习模型缺乏透明性和可解释性，仅能被视作黑箱，这限制了其广泛应用。专家需要理解AI模型的复杂行为及底层决策过程。可解释人工智能领域作为新兴领域，为AI模型的稳健、实用及可信部署提供了手段。现有多种针对图像分类任务的可解释人工智能技术，但图像分割的解释仍鲜有探索。本文通过适配近期可解释人工智能分类算法并使其适用于多类别图像分割（主要关注高分辨率卫星图像中建筑物分割），旨在弥补这一差距。为基准测试和比较所提出方法的性能，我们引入一种基于“熵”的新可解释人工智能评估方法及指标，用于衡量模型不确定性。传统可解释人工智能评估方法主要依赖将图像中的感兴趣区域反馈至预训练（效用）模型，再计算目标类别概率的平均变化。此类评估指标缺乏所需鲁棒性，我们证明利用熵监测模型在分割目标类别像素时的不确定性更为合适。期望本工作能为遥感领域图像分割及更多可解释人工智能研究铺平道路。