Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. But what exactly in the training data causes a model to make a certain prediction? We seek to answer this question by providing a language for describing how training data influences predictions, through a causal framework. Importantly, our framework bypasses the need to retrain expensive models and allows us to estimate causal effects based on observational data alone. Addressing the problem of extracting factual knowledge from pretrained language models (PLMs), we focus on simple data statistics such as co-occurrence counts and show that these statistics do influence the predictions of PLMs, suggesting that such models rely on shallow heuristics. Our causal framework and our results demonstrate the importance of studying datasets and the benefits of causality for understanding NLP models.

翻译：大量训练数据是当前最先进的NLP模型高性能的主要原因之一。但训练数据中的具体哪些因素会导致模型做出特定预测？我们旨在通过因果框架提供一种描述训练数据如何影响预测的语言来回答这一问题。重要的是，我们的框架无需重新训练昂贵的模型，仅基于观测数据即可估计因果效应。针对从预训练语言模型（PLMs）中提取事实知识的问题，我们聚焦于共现频率等简单数据统计量，并证明这些统计量确实会影响PLMs的预测，表明此类模型依赖浅层启发式策略。我们的因果框架和实验结果揭示了研究数据集的重要性以及因果关系在理解NLP模型中的优势。