In recent years, explainable methods for artificial intelligence (XAI) have tried to reveal and describe models' decision mechanisms in the case of classification tasks. However, XAI for semantic segmentation and in particular for single instances has been little studied to date. Understanding the process underlying automatic segmentation of single instances is crucial to reveal what information was used to detect and segment a given object of interest. In this study, we proposed two instance-level explanation maps for semantic segmentation based on SmoothGrad and Grad-CAM++ methods. Then, we investigated their relevance for the detection and segmentation of white matter lesions (WML), a magnetic resonance imaging (MRI) biomarker in multiple sclerosis (MS). 687 patients diagnosed with MS for a total of 4043 FLAIR and MPRAGE MRI scans were collected at the University Hospital of Basel, Switzerland. Data were randomly split into training, validation and test sets to train a 3D U-Net for MS lesion segmentation. We observed 3050 true positive (TP), 1818 false positive (FP), and 789 false negative (FN) cases. We generated instance-level explanation maps for semantic segmentation, by developing two XAI methods based on SmoothGrad and Grad-CAM++. We investigated: 1) the distribution of gradients in saliency maps with respect to both input MRI sequences; 2) the model's response in the case of synthetic lesions; 3) the amount of perilesional tissue needed by the model to segment a lesion. Saliency maps (based on SmoothGrad) in FLAIR showed positive values inside a lesion and negative in its neighborhood. Peak values of saliency maps generated for these four groups of volumes presented distributions that differ significantly from one another, suggesting a quantitative nature of the proposed saliency. Contextual information of 7mm around the lesion border was required for their segmentation.

翻译：近年来，人工智能可解释方法（XAI）致力于在分类任务中揭示和描述模型决策机制。然而，针对语义分割尤其是单实例的XAI研究迄今仍较为匮乏。理解单实例自动分割的底层过程，对于揭示模型如何检测和分割特定感兴趣目标所依赖的信息至关重要。本研究基于SmoothGrad和Grad-CAM++方法，提出了两种面向语义分割的实例级解释图。随后，我们探究了这些方法在白质病灶（WML）检测与分割中的有效性——WML是多发性硬化（MS）的磁共振成像（MRI）生物标志物。研究纳入瑞士巴塞尔大学医院收集的687例MS患者的4043次FLAIR和MPRAGE MRI扫描数据。数据随机划分为训练集、验证集和测试集，用于训练三维U-Net模型进行MS病灶分割。共观察到3050个真阳性（TP）、1818个假阳性（FP）和789个假阴性（FN）案例。通过开发基于SmoothGrad和Grad-CAM++的两种XAI方法，我们生成了语义分割的实例级解释图。研究内容包括：1）显著性图中梯度相对于两种MRI序列的分布特征；2）模型对合成病灶的响应机制；3）模型分割病灶所需的最小病灶周围组织范围。基于SmoothGrad的显著性图显示，FLAIR序列中病灶内部呈现正值，其邻域呈现负值。四组体积生成的显著性图峰值分布存在显著差异，表明所提显著性具有定量特性。病灶分割需要其边界周围7mm范围内的上下文信息。