Humans have a powerful and mysterious capacity to reason. By working through a series of purely mental steps, we can make inferences we would not be capable of making directly -- despite that fact that we get no additional data from the world. Similarly, large language models can perform better at complex tasks through chain-of-thought reasoning, where they generate intermediate steps before answering a question. We use language models to investigate the questions of when and why reasoning is helpful, testing the hypothesis that reasoning is effective when training data consisting of local clusters of variables that influence each other strongly. These training conditions enable the chaining of accurate local inferences in order to estimate relationships between variables that were not seen together in training. We train an autoregressive transformer on samples from joint distributions defined by Bayes nets, but only include a subset of all the variables in each sample. We compare language models' ability to match conditional probabilities both with and without intermediate reasoning steps, finding that intermediate steps help only when the training data is locally structured with respect to dependencies between variables. Furthermore, intermediate variables need to be relevant to the relationship between observed information and target inferences. Our results illustrate how the statistical structure of training data drives the effectiveness of reasoning step by step.

翻译：摘要：人类拥有强大而神秘的推理能力。通过一系列纯粹的思维步骤，我们能够做出原本无法直接得出的推断——尽管并未从外界获取额外数据。类似地，大型语言模型通过思维链推理（即在回答问题前先生成中间步骤），能在复杂任务中表现更佳。我们借助语言模型探究推理在何时、为何有效，并检验以下假设：当训练数据由相互强烈影响的局部变量簇构成时，推理尤为有效。这些训练条件使得语言模型能够通过串联准确的局部推断，来估计训练中未曾同时出现的变量间关系。我们基于贝叶斯网络定义的联合分布采样，训练自回归Transformer模型，但每个样本仅包含所有变量的子集。我们比较了语言模型在有、无中间推理步骤下匹配条件概率的能力，发现仅当训练数据在变量依赖关系上呈现局部结构时，中间步骤才起到帮助作用。此外，中间变量需与观测信息与目标推断之间的关系相关。我们的结果揭示了训练数据的统计结构如何驱动逐步推理的有效性。