Deep neural networks are increasingly used in natural language processing (NLP) models. However, the need to interpret and explain the results from complex algorithms are limiting their widespread adoption in regulated industries such as banking. There has been recent work on interpretability of machine learning algorithms with structured data. But there are only limited techniques for NLP applications where the problem is more challenging due to the size of the vocabulary, high-dimensional nature, and the need to consider textual coherence and language structure. This paper develops a methodology to compute SHAP values for local explainability of CNN-based text classification models. The approach is also extended to compute global scores to assess the importance of features. The results are illustrated on sentiment analysis of Amazon Electronic Review data.

翻译：深神经网络越来越多地用于自然语言处理模式(NLP),然而,解释和解释复杂算法结果的必要性限制了在银行等受监管行业广泛采用这些算法,最近还开展了关于机器学习算法与结构化数据的解释性的工作,但对于NLP应用程序而言,由于词汇的大小、高维度性质以及需要考虑文字一致性和语言结构,问题更具有挑战性,因此只有有限的技术在NLP应用程序中存在。本文开发了一种计算SHAP值的方法,用于计算CNN的文本分类模型的本地可解释性。该方法还扩展至计算全球得分,以评估特征的重要性。结果在亚马逊电子审查数据的情绪分析中作了说明。