In this paper, we investigate whether Large Language Models (LLMs) actively recall or retrieve their internal repositories of factual knowledge when faced with reasoning tasks. Through an analysis of LLMs' internal factual recall at each reasoning step via Knowledge Neurons, we reveal that LLMs fail to harness the critical factual associations under certain circumstances. Instead, they tend to opt for alternative, shortcut-like pathways to answer reasoning questions. By manually manipulating the recall process of parametric knowledge in LLMs, we demonstrate that enhancing this recall process directly improves reasoning performance whereas suppressing it leads to notable degradation. Furthermore, we assess the effect of Chain-of-Thought (CoT) prompting, a powerful technique for addressing complex reasoning tasks. Our findings indicate that CoT can intensify the recall of factual knowledge by encouraging LLMs to engage in orderly and reliable reasoning. Furthermore, we explored how contextual conflicts affect the retrieval of facts during the reasoning process to gain a comprehensive understanding of the factual recall behaviors of LLMs. Code and data will be available soon.

翻译：本文研究大型语言模型在面对推理任务时是否主动回忆或检索其内部事实知识库。通过知识神经元分析LLMs在每一推理步骤中的内部事实回忆过程，我们发现LLMs在某些情况下未能利用关键事实关联，反而倾向于选择替代性的捷径式路径来回答推理问题。通过人工操控LLMs中参数化知识的回忆过程，我们证明增强该回忆过程可直接提升推理性能，而抑制该过程则会导致显著性能下降。此外，我们评估了思维链提示这一解决复杂推理任务的有效技术的影响。研究结果表明，CoT能够通过促使LLMs进行有序可靠的推理来强化事实知识的回忆。进一步地，我们探究了上下文冲突如何影响推理过程中的事实检索，以全面理解LLMs的事实回忆行为。代码与数据即将公开。