Large language models rely on real-valued representations of text to make their predictions. These representations contain information learned from the data that the model has trained on, including knowledge of linguistic properties and forms of demographic bias, e.g., based on gender. A growing body of work has considered information about concepts such as these using orthogonal projections onto subspaces of the representation space. We contribute to this body of work by proposing a formal definition of intrinsic information in a subspace of a language model's representation space. We propose a counterfactual approach that avoids the failure mode of spurious correlations (Kumar et al., 2022) by treating components in the subspace and its orthogonal complement independently. We show that our counterfactual notion of information in a subspace is optimizing by an causal concept subspace. Furthermore, this intervention allows us to attempt concept controlled generation by manipulating the value of the conceptual component of a representation. Empirically, we find that R-LACE (Ravfogel et al., 2022) returns a one-dimensional subspace containing roughly half of total concept information under our framework. Our causal controlled intervention shows that, for at least one model, the subspace returned by R-LACE can be used to manipulate the concept value of the generated word with precision.

翻译：大型语言模型依赖文本的实值表示来进行预测。这些表示包含了模型从训练数据中学习到的信息，包括语言属性知识和基于性别等的人口统计学偏见。越来越多的研究通过将表示空间中的子空间进行正交投影来考察这些概念信息。我们通过提出语言模型表示空间中子空间内在信息的形式化定义，为这一研究领域做出贡献。我们提出了一种反事实方法，通过独立处理子空间及其正交补集中的分量，避免了虚假相关性（Kumar等人，2022）的失败模式。我们证明，子空间中信息的反事实概念可通过因果概念子空间进行优化。此外，这种干预使我们能够通过操纵表示中概念分量的值，尝试进行概念受控生成。实验发现，在我们的框架下，R-LACE（Ravfogel等人，2022）返回的一维子空间包含约一半的总概念信息。我们的因果受控干预表明，对于至少一个模型而言，R-LACE返回的子空间可用于精确操纵生成词的概念值。