To correctly use in-context information, language models (LMs) must bind entities to their attributes. For example, given a context describing a "green square" and a "blue circle", LMs must bind the shapes to their respective colors. We analyze LM representations and identify the binding ID mechanism: a general mechanism for solving the binding problem, which we observe in every sufficiently large model from the Pythia and LLaMA families. Using causal interventions, we show that LMs' internal activations represent binding information by attaching binding ID vectors to corresponding entities and attributes. We further show that binding ID vectors form a continuous subspace, in which distances between binding ID vectors reflect their discernability. Overall, our results uncover interpretable strategies in LMs for representing symbolic knowledge in-context, providing a step towards understanding general in-context reasoning in large-scale LMs.

翻译：为了正确利用上下文信息，语言模型（LMs）必须将实体与其属性绑定。例如，给定描述“绿色正方形”和“蓝色圆形”的上下文，语言模型必须将形状与各自的颜色绑定。我们分析了语言模型的表示并识别出绑定ID机制：一种解决绑定问题的通用机制，我们在Pythia和LLaMA家族的每个足够大的模型中均观察到了这一机制。通过因果干预，我们发现语言模型的内部激活通过将绑定ID向量附加到对应实体和属性上来表示绑定信息。我们进一步表明，绑定ID向量形成一个连续子空间，其中绑定ID向量之间的距离反映了其可辨别性。总体而言，我们的结果揭示了语言模型在上下文中表示符号知识的可解释策略，为理解大规模语言模型中的通用上下文推理迈出了一步。