In recent years, the development of large pretrained language models, such as BERT and GPT, significantly improved information extraction systems on various tasks, including relation classification. State-of-the-art systems are highly accurate on scientific benchmarks. A lack of explainability is currently a complicating factor in many real-world applications. Comprehensible systems are necessary to prevent biased, counterintuitive, or harmful decisions. We introduce semantic extents, a concept to analyze decision patterns for the relation classification task. Semantic extents are the most influential parts of texts concerning classification decisions. Our definition allows similar procedures to determine semantic extents for humans and models. We provide an annotation tool and a software framework to determine semantic extents for humans and models conveniently and reproducibly. Comparing both reveals that models tend to learn shortcut patterns from data. These patterns are hard to detect with current interpretability methods, such as input reductions. Our approach can help detect and eliminate spurious decision patterns during model development. Semantic extents can increase the reliability and security of natural language processing systems. Semantic extents are an essential step in enabling applications in critical areas like healthcare or finance. Moreover, our work opens new research directions for developing methods to explain deep learning models.

翻译：近年来，BERT和GPT等大型预训练语言模型的发展显著提升了信息抽取系统在多项任务上的性能，包括关系分类。当前最先进的系统在科学基准测试中具有高准确率，但在许多实际应用中，缺乏可解释性成为复杂因素。为避免有偏见、反直觉或有害的决策，需要具备可理解性的系统。我们提出语义广度的概念，用于分析关系分类任务的决策模式。语义广度是文本中对分类决策影响最大的部分。我们的定义使得人类和模型可通过类似程序确定语义广度。我们提供了一个标注工具和软件框架，以便便捷且可重复地确定人类和模型的语义广度。两者对比表明，模型倾向于从数据中学习捷径模式，而这类模式难以通过当前的可解释性方法（如输入缩减）检测到。我们的方法有助于在模型开发过程中检测并消除虚假决策模式。语义广度可提升自然语言处理系统的可靠性与安全性，并为在医疗或金融等关键领域应用奠定基础。此外，本文工作为开发深度学习模型解释方法开辟了新研究方向。