In recent years, explainable methods for artificial intelligence (XAI) have tried to reveal and describe models' decision mechanisms in the case of classification tasks. However, XAI for semantic segmentation and in particular for single instances has been little studied to date. Understanding the process underlying automatic segmentation of single instances is crucial to reveal what information was used to detect and segment a given object of interest. In this study, we proposed two instance-level explanation maps for semantic segmentation based on SmoothGrad and Grad-CAM++ methods. Then, we investigated their relevance for the detection and segmentation of white matter lesions (WML), a magnetic resonance imaging (MRI) biomarker in multiple sclerosis (MS). 687 patients diagnosed with MS for a total of 4043 FLAIR and MPRAGE MRI scans were collected at the University Hospital of Basel, Switzerland. Data were randomly split into training, validation and test sets to train a 3D U-Net for MS lesion segmentation. We observed 3050 true positive (TP), 1818 false positive (FP), and 789 false negative (FN) cases. We generated instance-level explanation maps for semantic segmentation, by developing two XAI methods based on SmoothGrad and Grad-CAM++. We investigated: 1) the distribution of gradients in saliency maps with respect to both input MRI sequences; 2) the model's response in the case of synthetic lesions; 3) the amount of perilesional tissue needed by the model to segment a lesion. Saliency maps (based on SmoothGrad) in FLAIR showed positive values inside a lesion and negative in its neighborhood. Peak values of saliency maps generated for these four groups of volumes presented distributions that differ significantly from one another, suggesting a quantitative nature of the proposed saliency. Contextual information of 7mm around the lesion border was required for their segmentation.

翻译：近年来，人工智能可解释性方法试图揭示和描述模型在分类任务中的决策机制。然而，针对语义分割特别是单个实例的可解释性研究迄今仍较为有限。理解单个实例自动分割的底层过程对于揭示检测和分割特定感兴趣对象时所使用的信息至关重要。本研究基于SmoothGrad和Grad-CAM++方法，提出了两种用于语义分割的实例级解释图。随后，我们探究了这些方法在白质病变检测与分割中的适用性——白质病变是多发性硬化症中一种磁共振成像生物标志物。研究收集了巴塞尔大学医院687名确诊多发性硬化症患者的4043例FLAIR和MPRAGE磁共振扫描数据。数据被随机划分为训练集、验证集和测试集，用于训练三维U-Net进行多发性硬化病灶分割。我们观察到3050例真阳性、1818例假阳性和789例假阴性病例。通过开发两种基于SmoothGrad和Grad-CAM++的可解释性方法，我们生成了语义分割的实例级解释图。具体研究内容包括：1）显著性图中梯度在两种输入磁共振序列中的分布特征；2）模型对合成病灶的响应特性；3）模型分割病灶所需的病灶周围组织范围。基于SmoothGrad的FLAIR序列显著性图显示病灶内部呈正值而邻域呈负值。四组数据生成的显著性图峰值分布呈现显著差异，表明所提显著性具有定量特性。病灶分割需要边界周围7mm范围内的上下文信息。