Developing explainability methods for Natural Language Processing (NLP) models is a challenging task, for two main reasons. First, the high dimensionality of the data (large number of tokens) results in low coverage and in turn small contributions for the top tokens, compared to the overall model performance. Second, owing to their textual nature, the input variables, after appropriate transformations, are effectively binary (presence or absence of a token in an observation), making the input-output relationship difficult to understand. Common NLP interpretation techniques do not have flexibility in resolution, because they usually operate at word-level and provide fully local (message level) or fully global (over all messages) summaries. The goal of this paper is to create more flexible model explainability summaries by segments of observation or clusters of words that are semantically related to each other. In addition, we introduce a root cause analysis method for NLP models, by analyzing representative False Positive and False Negative examples from different segments. At the end, we illustrate, using a Yelp review data set with three segments (Restaurant, Hotel, and Beauty), that exploiting group/cluster structures in words and/or messages can aid in the interpretation of decisions made by NLP models and can be utilized to assess the model's sensitivity or bias towards gender, syntax, and word meanings.

翻译：开发自然语言处理模型的可解释性方法是一项具有挑战性的任务，主要原因有两点。首先，数据的高维性（大量词元）导致高覆盖率词元数量较少，进而使得关键词元相对于模型整体性能的贡献度偏低。其次，由于输入变量本质为文本形式，经过适当变换后，其实际为二元变量（观测样本中词元的存在与否），这使得输入输出关系难以理解。常见的自然语言处理解释技术缺乏分辨率灵活性，因为它们通常仅在词级别运作，并提供完全局部（信息级别）或完全全局（所有信息）的汇总。本文旨在通过观测片段或语义相关的词簇，为模型可解释性创建更灵活的汇总方式。此外，我们引入了一种针对自然语言处理模型的根本原因分析方法，即分析不同片段中具有代表性的假正例与假负例样本。最后，我们使用包含三个片段（餐厅、酒店和美容）的Yelp评论数据集证明：利用词和/或消息的群组/簇结构，有助于解释自然语言处理模型做出的决策，并可被用于评估模型对性别、句法和词义的敏感性或偏差。