Pre-trained language models (PLMs) like BERT are being used for almost all language-related tasks, but interpreting their behavior still remains a significant challenge and many important questions remain largely unanswered. In this work, we re-purpose a sentence editing dataset, where faithful high-quality human rationales can be automatically extracted and compared with extracted model rationales, as a new testbed for interpretability. This enables us to conduct a systematic investigation on an array of questions regarding PLMs' interpretability, including the role of pre-training procedure, comparison of rationale extraction methods, and different layers in the PLM. The investigation generates new insights, for example, contrary to the common understanding, we find that attention weights correlate well with human rationales and work better than gradient-based saliency in extracting model rationales. Both the dataset and code are available at https://github.com/samuelstevens/sentence-editing-interpretability to facilitate future interpretability research.

翻译：在这项工作中,我们重新启用了句子编辑数据集,将忠实的高质量人的理由自动提取出来,并与提取的模型原理进行比较,作为解释性的新测试台。这使我们能够对有关PLM的可解释性的一系列问题进行系统调查,包括培训前程序的作用、理由提取方法的比较,以及PLM的不同层次。例如,调查产生了新的洞察力,例如,与共同的理解相反,我们发现,在提取模型原理时,注意力与人的理由和工作比基于梯度的显著原理都好。 https://github.com/samuelstevens/sentence-edit-eding-interpretyable都提供数据集和代码,以便利未来的可解释性研究。