Pre-trained language models (LMs) are able to perform complex reasoning without explicit fine-tuning. To understand how pre-training with a next-token prediction objective contributes to the emergence of such reasoning capability, we propose that we can view an LM as deriving new conclusions by aggregating indirect reasoning paths seen at pre-training time. We found this perspective effective in two important cases of reasoning: logic reasoning with knowledge graphs (KGs) and math reasoning with math word problems (MWPs). More specifically, we formalize the reasoning paths as random walk paths on the knowledge/reasoning graphs. Analyses of learned LM distributions suggest that a weighted sum of relevant random walk path probabilities is a reasonable way to explain how LMs reason. Experiments and analysis on multiple KG and MWP datasets reveal the effect of training on random walk paths and suggest that augmenting unlabeled random walk reasoning paths can improve real-world multi-step reasoning performance. code: https://github.com/WANGXinyiLinda/LM_random_walk

翻译：预训练语言模型在没有明确微调的情况下能够执行复杂推理。为理解以预测下一词为目标的预训练如何催生这种推理能力，我们提出可将语言模型视为通过聚合预训练阶段所见到的间接推理路径来推导新结论。我们发现在两种重要推理场景——知识图谱逻辑推理与数学应用题推理——中，这一视角均十分有效。具体而言，我们将推理路径形式化为知识/推理图上的随机游走路径。对学习到的语言模型分布的分析表明，相关随机游走路径概率的加权和是解释语言模型推理机制的合理方式。在多个知识图谱与数学应用题数据集上的实验与分析揭示了随机游走路径训练的作用，并表明增强未标注的随机游走推理路径可提升实际多步推理性能。代码：https://github.com/WANGXinyiLinda/LM_random_walk