Faithful and robust pronoun use is important for fair and coherent generations, yet large language models largely fail when multiple referents use different pronouns. To study the interplay of reasoning, repetition, and bias in this task, prior work relies exclusively on behavioural approaches, which may not reflect a model's internal workings. Therefore, we provide a mechanistic, model-internal perspective on pronoun fidelity, testing whether three mechanisms -- group entity binding (G), recency bias (R), and stereotypical bias (S) -- are causally implemented across several SOTA language models. Using Boundless Distributed Alignment Search, we find all three coexist as causal subspaces distributed across network depth. No single mechanism fully explains model behaviour, but a combination of the three consistently accounts for 91-99.5%. An attention head analysis further reveals two competing copying routes; group binding and stereotype share a localized concept-level route that retrieves a bound occupation-pronoun unit, while recency uses a distributed token-level route that repeats surface forms. In sum, pronoun fidelity arises from competition between simultaneously active causal subspaces.

翻译：摘要：代词使用的忠实性与鲁棒性对于生成公平且连贯的文本至关重要，然而当多个指代对象使用不同代词时，大型语言模型常常出现错误。为探究该任务中推理、重复与偏差之间的相互作用，先前研究仅依赖行为学方法，这未必能反映模型内部运作机制。因此，我们从模型内部的机制视角出发研究代词忠实性，检验三种机制——组实体绑定（G）、近因偏差（R）与刻板印象偏差（S）——是否在多款最先进语言模型中被因果性地实现。通过无界分布式对齐搜索，我们发现三者共存为分布在网络深度中的因果子空间。没有任何单一机制能完全解释模型行为，但三者组合始终可解释91-99.5%的现象。注意力头分析进一步揭示了两种竞争性复制路径：组绑定与刻板印象共享一个局部化概念级路径，负责检索已绑定的职业-代词单元；而近因则利用分布式词元级路径重复表面形式。综上所述，代词忠实性源于同时激活的因果子空间之间的竞争。