Despite the recent success of large pretrained language models (LMs) on a variety of prompting tasks, these models can be alarmingly brittle to small changes in inputs or application contexts. To better understand such behavior and motivate the design of more robust LMs, we propose a general experimental framework, CALM (Competence-based Analysis of Language Models), where targeted causal interventions are utilized to damage an LM's internal representation of various linguistic properties in order to evaluate its use of each representation in performing a given task. We implement these interventions as gradient-based adversarial attacks, which (in contrast to prior causal probing methodologies) are able to target arbitrarily-encoded representations of relational properties, and carry out a case study of this approach to analyze how BERT-like LMs use representations of several relational properties in performing associated relation prompting tasks. We find that, while the representations LMs leverage in performing each task are highly entangled, they may be meaningfully interpreted in terms of the tasks where they are most utilized; and more broadly, that CALM enables an expanded scope of inquiry in LM analysis that may be useful in predicting and explaining weaknesses of existing LMs.

翻译：尽管大型预训练语言模型（LMs）在各类提示任务中近期取得了成功，但这些模型在输入或应用情境的微小变化下可能表现出令人担忧的脆弱性。为了深入理解此类行为并推动鲁棒性更强的LM设计，我们提出一个通用实验框架CALM（语言模型的能力基准分析），通过定向因果干预破坏LM对多种语言属性的内部表征，从而评估模型在特定任务中运用每种表征的方式。我们以基于梯度的对抗攻击实现这些干预（与先前的因果探针方法不同），该方法能够针对关系属性任意编码的表征，并通过案例研究分析BERT类语言模型在执行关联关系提示任务时如何利用多种关系属性的表征。研究发现：尽管LM在执行每项任务时使用的表征高度纠缠，但可通过其最常被调用的任务进行有意义的解释；更广泛而言，CALM可扩展LM分析的研究范围，有助于预测和解释现有LM的弱点。