eXplainable Artificial Intelligence (XAI) has emerged as an essential requirement when dealing with mission-critical applications, ensuring transparency and interpretability of the employed black box AI models. The significance of XAI spans various domains, from healthcare to finance, where understanding the decision-making process of deep learning algorithms is essential. Most AI-based computer vision models are often black boxes; hence, providing explainability of deep neural networks in image processing is crucial for their wide adoption and deployment in medical image analysis, autonomous driving, and remote sensing applications. Recently, several XAI methods for image classification tasks have been introduced. On the contrary, image segmentation has received comparatively less attention in the context of explainability, although it is a fundamental task in computer vision applications, especially in remote sensing. Only some research proposes gradient-based XAI algorithms for image segmentation. This paper adapts the recent gradient-free Sobol XAI method for semantic segmentation. To measure the performance of the Sobol method for segmentation, we propose a quantitative XAI evaluation method based on a learnable noise model. The main objective of this model is to induce noise on the explanation maps, where higher induced noise signifies low accuracy and vice versa. A benchmark analysis is conducted to evaluate and compare performance of three XAI methods, including Seg-Grad-CAM, Seg-Grad-CAM++ and Seg-Sobol using the proposed noise-based evaluation technique. This constitutes the first attempt to run and evaluate XAI methods using high-resolution satellite images.

翻译：可解释人工智能（XAI）已成为处理关键任务应用时的基本要求，旨在确保所用黑箱AI模型的透明性与可解释性。XAI的重要性涵盖从医疗到金融等多个领域，在这些领域中，理解深度学习算法的决策过程至关重要。大多数基于AI的计算机视觉模型通常属于黑箱模型，因此，在图像处理中提供深度神经网络的可解释性对于其在医学图像分析、自动驾驶及遥感应用中的广泛采用与部署至关重要。近年来，针对图像分类任务已引入多种XAI方法。相比之下，图像分割在可解释性方面受到的关注相对较少，尽管它是计算机视觉应用（尤其是遥感领域）中的基础任务。目前仅有少量研究提出了基于梯度的XAI图像分割算法。本文旨在将最新的无梯度Sobol XAI方法适配至语义分割任务。为衡量Sobol方法在分割任务中的性能，我们提出了一种基于可学习噪声模型的定量XAI评估方法。该模型的核心目标是在解释图上引入噪声，其中较高的引入噪声表示较低的准确性，反之亦然。通过基准分析，我们使用所提出的基于噪声的评估技术，对包括Seg-Grad-CAM、Seg-Grad-CAM++及Seg-Sobol在内的三种XAI方法进行了性能评估与比较。这是首次利用高分辨率卫星图像对XAI方法进行运行与评估的尝试。