Logic reasoning has been critically needed in problem-solving and decision-making. Although Language Models (LMs) have demonstrated capabilities of handling multiple reasoning tasks (e.g., commonsense reasoning), their ability to reason complex mathematical problems, specifically propositional logic, remains largely underexplored. This lack of exploration can be attributed to the limited availability of annotated corpora. Here, we present a well-labeled propositional logic corpus, LogicPrpBank, containing 7093 Propositional Logic Statements (PLSs) across six mathematical subjects, to study a brand-new task of reasoning logical implication and equivalence. We benchmark LogicPrpBank with widely-used LMs to show that our corpus offers a useful resource for this challenging task and there is ample room for model improvement.

翻译：逻辑推理在问题求解和决策过程中至关重要。尽管语言模型（LMs）已展现出处理多重推理任务（如常识推理）的能力，但它们在复杂数学问题推理（特别是命题逻辑）方面的能力仍未得到充分探索。这种探索不足可归因于标注语料库的匮乏。为此，我们提出了一个标注完善的命题逻辑语料库LogicPrpBank，包含涵盖六个数学学科的7093个命题逻辑语句（PLSs），用于研究逻辑蕴含与等价推理这一全新任务。我们采用广泛使用的语言模型对LogicPrpBank进行基准测试，结果表明本语料库为该挑战性任务提供了有价值的资源，且模型仍有较大的改进空间。