Understanding why a neural network model makes certain decisions can be as important as the inference performance. Various methods have been proposed to help practitioners explain the prediction of a neural network model, of which Shapley values are most popular. SHAP package is a leading implementation of Shapley values to explain neural networks implemented in TensorFlow or PyTorch but lacks cross-platform support, one-shot deployment and is highly inefficient. To address these problems, we present the ONNXExplainer, which is a generic framework to explain neural networks using Shapley values in the ONNX ecosystem. In ONNXExplainer, we develop its own automatic differentiation and optimization approach, which not only enables One-Shot Deployment of neural networks inference and explanations, but also significantly improves the efficiency to compute explanation with less memory consumption. For fair comparison purposes, we also implement the same optimization in TensorFlow and PyTorch and measure its performance against the current state of the art open-source counterpart, SHAP. Extensive benchmarks demonstrate that the proposed optimization approach improves the explanation latency of VGG19, ResNet50, DenseNet201, and EfficientNetB0 by as much as 500%.

翻译：理解神经网络模型做出特定决策的原因，与模型的推理性能同等重要。已有多种方法被提出以帮助从业者解释神经网络的预测结果，其中沙普利值最为流行。SHAP包是基于沙普利值解释TensorFlow或PyTorch神经网络的主要实现方案，但其缺乏跨平台支持、无法实现一次性部署，且效率极低。为解决这些问题，我们提出了ONNXExplainer——一种在ONNX生态系统中利用沙普利值解释神经网络的通用框架。ONNXExplainer开发了其自身的自动微分与优化方法，这不仅实现了神经网络推理与解释的一次性部署，还通过降低内存消耗显著提升了计算解释的效率。为公平比较，我们亦在TensorFlow和PyTorch中实现了相同的优化方法，并将其性能与当前最先进的开源方案SHAP进行对比。广泛基准测试表明，所提优化方法使VGG19、ResNet50、DenseNet201及EfficientNetB0的解释延迟最高提升了500%。