Causal probing aims to analyze foundation models by examining how intervening on their representation of various latent properties impacts their outputs. Recent works have cast doubt on the theoretical basis of several leading causal probing methods, but it has been unclear how to systematically evaluate the effectiveness of these methods in practice. To address this, we define two key causal probing desiderata: completeness (how thoroughly the representation of the target property has been transformed) and selectivity (how little non-targeted properties have been impacted). We find that there is an inherent tradeoff between the two, which we define as reliability, their harmonic mean. We introduce an empirical analysis framework to measure and evaluate these quantities, allowing us to make the first direct comparisons between different families of leading causal probing methods (e.g., linear vs. nonlinear, or concept removal vs. counterfactual interventions). We find that: (1) all methods show a clear tradeoff between completeness and selectivity; (2) more complete and reliable methods have a greater impact on LLM behavior; and (3) nonlinear interventions are almost always more reliable than linear interventions.

翻译：因果探测旨在通过干预基础模型对各类潜在属性的表征来考察其输出如何变化，从而分析这些模型。近期研究对几种主流因果探测方法的理论基础提出了质疑，但如何系统评估这些方法在实际中的有效性尚不明确。为解决此问题，我们定义了两个关键的因果探测需求：完备性（目标属性表征被转换的彻底程度）与选择性（非目标属性受影响的微小程度）。我们发现二者之间存在固有的权衡关系，我们将其定义为可靠性——即二者的调和平均数。我们提出了一个实证分析框架来量化和评估这些指标，从而首次实现了对不同类型主流因果探测方法（例如线性与非线性干预，或概念移除与反事实干预）的直接比较。研究发现：（1）所有方法均表现出完备性与选择性间的明显权衡；（2）更完备且更可靠的方法对大型语言模型行为的影响更大；（3）非线性干预几乎总是比线性干预更可靠。