Human logic has gradually shifted from intuition-driven inference to rigorous formal systems. Motivated by recent advances in large language models (LLMs), we explore whether LLMs exhibit a similar evolution in the underlying logical framework. Using existential import as a probe, we for evaluate syllogism under traditional and modern logic. Through extensive experiments of testing SOTA LLMs on a new syllogism dataset, we have some interesting findings: (i) Model size scaling promotes the shift toward modern logic; (ii) Thinking serves as an efficient accelerator beyond parameter scaling; (iii) the Base model plays a crucial role in determining how easily and stably this shift can emerge. Beyond these core factors, we conduct additional experiments for in-depth analysis of properties of current LLMs on syllogistic reasoning.
翻译:人类逻辑已逐渐从直觉驱动的推理转向严谨的形式化系统。受近期大型语言模型(LLMs)进展的启发,我们探究LLMs是否在其底层逻辑框架中表现出类似的演化规律。我们以存在性引入为探针,分别基于传统逻辑和现代逻辑评估三段论推理。通过在新型三段论数据集上对前沿LLMs进行大量实验,我们获得若干重要发现:(1)模型规模扩展促进向现代逻辑的转变;(2)思维链机制可作为超越参数扩展的高效加速器;(3)基础模型对决定该转变能否稳定涌现具有关键作用。除这些核心因素外,我们通过补充实验深入分析了当前LLMs在三段论推理任务中的特性。