Beyond the great cognitive powers showcased by language models, it is crucial to scrutinize whether their reasoning capabilities stem from strong generalization or merely exposure to relevant data. As opposed to constructing increasingly complex logic, this paper probes into the boolean logic, the root capability of a logical reasoner. We find that any pre-trained language models even including large language models only behave like a random selector in the face of multi-nested boolean logic, a task that humans can handle with ease. To empower language models with this fundamental capability, this paper proposes a new self-supervised learning method \textit{Curriculum Logical Reasoning} (\textsc{Clr}), where we augment the training data with nested boolean logic chain step-by-step, and program the training from simpler logical patterns gradually to harder ones. This new training paradigm allows language models to effectively generalize to much harder and longer-hop logic, which can hardly be learned through naive training. Furthermore, we show that boolean logic is a great foundation for improving the subsequent general logical tasks.

翻译：超越语言模型展现的卓越认知能力，关键在于审视其推理能力是源于强大的泛化能力，还是仅来自相关数据的被动暴露。本文不试图构建日益复杂的逻辑体系，而是直指逻辑推理器的根基——布尔逻辑。我们发现，任何预训练语言模型，包括大型语言模型在内，面对人类可轻松处理的嵌套布尔逻辑任务时，其表现仅如同随机选择器。为赋予语言模型这一基础能力，本文提出一种新型自监督学习方法——课程逻辑推理（\textsc{Clr}）。该方法通过逐步增强训练数据中的嵌套布尔逻辑链，并按照从简单到复杂的逻辑模式编排训练进程。这一新训练范式使语言模型能有效泛化至难度更高、跳数更长的逻辑推理，而这些通过朴素训练几乎无法习得。此外，我们论证了布尔逻辑可作为提升后续通用逻辑任务的坚实基础。