Despite superior reasoning prowess demonstrated by Large Language Models (LLMs) with Chain-of-Thought (CoT) prompting, a lack of understanding prevails around the internal mechanisms of the models that facilitate CoT generation. This work investigates the neural sub-structures within LLMs that manifest CoT reasoning from a mechanistic point of view. From an analysis of Llama-2 7B applied to multistep reasoning over fictional ontologies, we demonstrate that LLMs deploy multiple parallel pathways of answer generation for step-by-step reasoning. These parallel pathways provide sequential answers from the input question context as well as the generated CoT. We observe a functional rift in the middle layers of the LLM. Token representations in the initial half remain strongly biased towards the pretraining prior, with the in-context prior taking over in the later half. This internal phase shift manifests in different functional components: attention heads that write the answer token appear in the later half, attention heads that move information along ontological relationships appear in the initial half, and so on. To the best of our knowledge, this is the first attempt towards mechanistic investigation of CoT reasoning in LLMs.

翻译：尽管大型语言模型（LLMs）在链式推理（CoT）提示下展现出卓越的推理能力，但人们对模型内部促进CoT生成的机制仍缺乏理解。本研究从机械论视角出发，探究LLMs中体现CoT推理的神经子结构。通过对应用于虚构本体多步推理的Llama-2 7B模型进行分析，我们证明LLMs在逐步推理过程中部署了多条并行的答案生成路径。这些并行路径既从输入问题上下文也从生成的CoT中提供顺序答案。我们观察到LLM中间层存在功能分界：前半部分的词元表示强烈偏向预训练先验，而后半部分则由上下文先验主导。这种内部相位转换体现在不同功能组件中：写入答案词元的注意力头出现在后半部分，而沿本体关系传递信息的注意力头则出现在前半部分。据我们所知，这是首次对LLMs中CoT推理进行机械论机制探索的研究尝试。