We study syllogistic reasoning in LLMs from the logical and natural language perspectives. In process, we explore fundamental reasoning capabilities of the LLMs and the direction this research is moving forward. To aid in our studies, we use 14 large language models and investigate their syllogistic reasoning capabilities in terms of symbolic inferences as well as natural language understanding. Even though this reasoning mechanism is not a uniform emergent property across LLMs, the perfect symbolic performances in certain models make us wonder whether LLMs are becoming more and more formal reasoning mechanisms, rather than making explicit the nuances of human reasoning.

翻译：本研究从逻辑与自然语言双重视角探讨大语言模型的三段论推理能力。在此过程中，我们深入考察大语言模型的基础推理能力及其研究发展趋势。为支撑研究，我们选用14个大型语言模型，从符号推理与自然语言理解两个维度系统检验其三段论推理性能。尽管该推理机制并非所有大语言模型普遍涌现的特性，但部分模型在符号推理上的完美表现促使我们思考：大语言模型是否正日益演化为形式化推理机制，而非显式模拟人类推理的微妙特征。