In this paper, we investigate whether Large Language Models (LLMs) actively recall or retrieve their internal repositories of factual knowledge when faced with reasoning tasks. Through an analysis of LLMs' internal factual recall at each reasoning step via Knowledge Neurons, we reveal that LLMs fail to harness the critical factual associations under certain circumstances. Instead, they tend to opt for alternative, shortcut-like pathways to answer reasoning questions. By manually manipulating the recall process of parametric knowledge in LLMs, we demonstrate that enhancing this recall process directly improves reasoning performance whereas suppressing it leads to notable degradation. Furthermore, we assess the effect of Chain-of-Thought (CoT) prompting, a powerful technique for addressing complex reasoning tasks. Our findings indicate that CoT can intensify the recall of factual knowledge by encouraging LLMs to engage in orderly and reliable reasoning. Furthermore, we explored how contextual conflicts affect the retrieval of facts during the reasoning process to gain a comprehensive understanding of the factual recall behaviors of LLMs. Code and data will be available soon.

翻译：本文研究大型语言模型在面对推理任务时是否会主动回忆或检索其内部存储的事实知识。通过知识神经元分析LLM在每一步推理中的内部事实回忆过程，我们发现LLM在某些情况下未能利用关键的事实关联，反而倾向于选择替代性的、类似捷径的路径来回答推理问题。通过手动操控LLM中参数化知识的回忆过程，我们证明增强这一回忆过程可直接提升推理性能，而抑制该过程则会导致显著性能下降。此外，我们评估了思维链提示这一解决复杂推理任务的有效技术的影响。研究结果表明，CoT能够通过促使LLM进行有序可靠的推理来强化事实知识的回忆。我们还深入探究了上下文冲突如何影响推理过程中的事实检索，从而全面理解LLM的事实回忆行为。代码与数据即将公开。