Explainable artificial intelligence (XAI) methods shed light on the predictions of deep neural networks (DNNs). Several different approaches exist and have partly already been successfully applied in climate science. However, the often missing ground truth explanations complicate their evaluation and validation, subsequently compounding the choice of the XAI method. Therefore, in this work, we introduce XAI evaluation in the context of climate research and assess different desired explanation properties, namely, robustness, faithfulness, randomization, complexity, and localization. To this end we build upon previous work and train a multi-layer perceptron (MLP) and a convolutional neural network (CNN) to predict the decade based on annual-mean temperature maps. Next, multiple local XAI methods are applied and their performance is quantified for each evaluation property and compared against a baseline test. Independent of the network type, we find that the XAI methods Integrated Gradients, Layer-wise relevance propagation, and InputGradients exhibit considerable robustness, faithfulness, and complexity while sacrificing randomization. The opposite is true for Gradient, SmoothGrad, NoiseGrad, and FusionGrad. Notably, explanations using input perturbations, such as SmoothGrad and Integrated Gradients, do not improve robustness and faithfulness, contrary to previous claims. Overall, our experiments offer a comprehensive overview of different properties of explanation methods in the climate science context and supports users in the selection of a suitable XAI method.

翻译：可解释人工智能（XAI）方法揭示深度神经网络（DNNs）的预测机制。目前存在多种不同的方法，其中部分已在气候科学中得到成功应用。然而，由于解释往往缺乏真值，这使评估与验证变得复杂，进而加剧了XAI方法选择的难度。因此，本研究在气候研究背景下引入XAI评估，并考察不同的期望解释特性，即鲁棒性、忠实性、随机化、复杂性与局部性。为此，我们基于先前工作，训练了一个多层感知机（MLP）和一个卷积神经网络（CNN），用于根据年均温度图预测年代。随后，应用多种局部XAI方法，定量评估其在每个特性上的表现，并与基线测试进行对比。与网络类型无关，我们发现XAI方法Integrated Gradients、层相关传播和InputGradients在鲁棒性、忠实性和复杂性方面表现显著，但牺牲了随机化性；而Gradient、SmoothGrad、NoiseGrad和FusionGrad则表现相反。值得注意的是，使用输入扰动（如SmoothGrad和Integrated Gradients）的解释并未如先前声明那样提升鲁棒性和忠实性。总体而言，我们的实验全面概述了气候科学背景下解释方法的不同特性，并支持用户选择合适的XAI方法。