Non-deductive reasoning, encompassing inductive and abductive reasoning, is essential in addressing complex real-world questions. One key feature of inductive and abductive reasoning is that there are many valid hypotheses; the simplest ones (those that adhere to Occam's Razor) are often most useful. However, this aspect is ignored in recent work that evaluates the non-deductive reasoning capabilities of large language models (LLMs). This work fills this gap, focusing on understanding whether the inductive and abductive reasoning capabilities of LLMs adhere to Occam's Razor, while also examining the correctness of their reasoning. To accomplish this goal, we introduce a framework to synthetically generate reasoning questions that (a) require inductive reasoning and abductive reasoning simultaneously; (b) is readily extended to produce any abductive/inductive reasoning question expressible in first-order logic. The task for the intelligent agent is to produce hypotheses to explain observations under a given world model. We also propose a new automated metric to assess whether hypotheses quantitatively adhere to Occam's Razor; those hypotheses that are correct and simplest are considered high-quality. Our findings on state-of-the-art LLMs suggest that LLMs can perform inductive and abductive reasoning in simple scenarios, but struggle with complex world models and with producing high-quality hypotheses, even with popular reasoning-enhancing techniques such as in-context learning and RLVR.

翻译：非演绎推理（涵盖归纳推理与溯因推理）是解决复杂现实问题的核心能力。这类推理的关键特征在于存在多种有效假设，而最简假设（遵循奥卡姆剃刀原则者）往往最具实用价值。然而，近期评估大语言模型非演绎推理能力的研究普遍忽视了这一维度。本研究旨在弥补该空白，重点探究大语言模型的归纳与溯因推理能力是否遵循奥卡姆剃刀原则，同时考察其推理正确性。为此，我们提出一个合成推理题目的框架：该框架可生成同时需要演绎与溯因推理的题目，并易于扩展至任意可用一阶逻辑表达的溯因/归纳推理问题。智能体的任务是在给定世界模型下生成解释观测数据的假设。我们同时提出新的自动化评估指标，用于量化假设对奥卡姆剃刀原则的遵循程度；正确且最简的假设被视为高质量假设。针对前沿大语言模型的实验表明，这些模型虽能在简单场景中执行归纳与溯因推理，但在复杂世界模型建模及生成高质量假设方面仍存在困难——即便采用上下文学习、强化学习验证等主流推理增强技术亦不例外。