eXplainable Artificial Intelligence (XAI) has emerged as an essential requirement when dealing with mission-critical applications, ensuring transparency and interpretability of the employed black box AI models. The significance of XAI spans various domains, from healthcare to finance, where understanding the decision-making process of deep learning algorithms is essential. Most AI-based computer vision models are often black boxes; hence, providing explainability of deep neural networks in image processing is crucial for their wide adoption and deployment in medical image analysis, autonomous driving, and remote sensing applications. Recently, several XAI methods for image classification tasks have been introduced. On the contrary, image segmentation has received comparatively less attention in the context of explainability, although it is a fundamental task in computer vision applications, especially in remote sensing. Only some research proposes gradient-based XAI algorithms for image segmentation. This paper adapts the recent gradient-free Sobol XAI method for semantic segmentation. To measure the performance of the Sobol method for segmentation, we propose a quantitative XAI evaluation method based on a learnable noise model. The main objective of this model is to induce noise on the explanation maps, where higher induced noise signifies low accuracy and vice versa. A benchmark analysis is conducted to evaluate and compare performance of three XAI methods, including Seg-Grad-CAM, Seg-Grad-CAM++ and Seg-Sobol using the proposed noise-based evaluation technique. This constitutes the first attempt to run and evaluate XAI methods using high-resolution satellite images.

翻译：可解释人工智能（XAI）已成为处理关键任务应用时的基本要求，它确保了所采用的黑箱AI模型的透明性和可解释性。XAI的重要性涵盖从医疗到金融等多个领域，在这些领域中，理解深度学习算法的决策过程至关重要。大多数基于AI的计算机视觉模型往往是黑箱模型；因此，在图像处理中提供深度神经网络的可解释性，对于其在医学图像分析、自动驾驶和遥感应用中的广泛采用和部署至关重要。近年来，针对图像分类任务已引入了多种XAI方法。相比之下，图像分割在可解释性方面受到的关注相对较少，尽管它是计算机视觉应用（尤其是遥感领域）中的一项基础任务。仅有少数研究提出了用于图像分割的基于梯度的XAI算法。本文针对语义分割任务调整了最近提出的无梯度Sobol XAI方法。为了衡量Sobol方法在分割中的性能，我们提出了一种基于可学习噪声模型的定量XAI评估方法。该模型的主要目标是在解释图上引入噪声，其中较高的引入噪声表示较低的准确性，反之亦然。通过基准分析，我们使用所提出的基于噪声的评估技术，评估并比较了三种XAI方法（包括Seg-Grad-CAM、Seg-Grad-CAM++和Seg-Sobol）的性能。这是首次尝试使用高分辨率卫星图像运行和评估XAI方法。