Explainability is a key component in many applications involving deep neural networks (DNNs). However, current explanation methods for DNNs commonly leave it to the human observer to distinguish relevant explanations from spurious noise. This is not feasible anymore when going from easily human-accessible data such as images to more complex data such as genome sequences. To facilitate the accessibility of DNN outputs from such complex data and to increase explainability, we present a modification of the widely used explanation method layer-wise relevance propagation. Our approach enforces sparsity directly by pruning the relevance propagation for the different layers. Thereby, we achieve sparser relevance attributions for the input features as well as for the intermediate layers. As the relevance propagation is input-specific, we aim to prune the relevance propagation rather than the underlying model architecture. This allows to prune different neurons for different inputs and hence, might be more appropriate to the local nature of explanation methods. To demonstrate the efficacy of our method, we evaluate it on two types of data: images and genome sequences. We show that our modification indeed leads to noise reduction and concentrates relevance on the most important features compared to the baseline.

翻译：可解释性是涉及深度神经网络（DNN）的众多应用中的关键组成部分。然而，当前针对深度神经网络的解释方法通常需要人类观察者自行区分相关解释与虚假噪声。当处理对象从图像等易于人类理解的数据扩展到基因组序列等更复杂的数据时，这种方法便不再可行。为提升此类复杂数据中深度神经网络输出的可访问性并增强可解释性，我们对广泛使用的逐层相关性传播解释方法进行了改进。该方法通过剪裁不同层的相关性传播路径直接实现稀疏化，从而在输入特征层和中间层均获得更稀疏的相关性归因。由于相关性传播具有输入特异性，我们的目标是对传播过程而非底层模型架构进行剪枝。这使得不同输入可剪裁不同神经元，因而更契合解释方法的局部特性。为验证方法的有效性，我们在图像和基因组序列两类数据上进行了评估。实验表明，相较于基线方法，本改进方案确实实现了噪声抑制，并将相关性集中于最重要的特征上。