Combining deep learning with symbolic logic reasoning aims to capitalize on the success of both fields and is drawing increasing attention. Inspired by DeepLogic, an end-to-end model trained to perform inference on logic programs, we introduce IMA-GloVe-GA, an iterative neural inference network for multi-step reasoning expressed in natural language. In our model, reasoning is performed using an iterative memory neural network based on RNN with a gated attention mechanism. We evaluate IMA-GloVe-GA on three datasets: PARARULES, CONCEPTRULES V1 and CONCEPTRULES V2. Experimental results show DeepLogic with gated attention can achieve higher test accuracy than DeepLogic and other RNN baseline models. Our model achieves better out-of-distribution generalisation than RoBERTa-Large when the rules have been shuffled. Furthermore, to address the issue of unbalanced distribution of reasoning depths in the current multi-step reasoning datasets, we develop PARARULE-Plus, a large dataset with more examples that require deeper reasoning steps. Experimental results show that the addition of PARARULE-Plus can increase the model's performance on examples requiring deeper reasoning depths. The source code and data are available at https://github.com/Strong-AI-Lab/Multi-Step-Deductive-Reasoning-Over-Natural-Language.

翻译：结合深度学习与符号逻辑推理旨在利用两个领域的成功经验，并日益受到关注。受DeepLogic（一种训练用于对逻辑程序进行推理的端到端模型）启发，我们提出了IMA-GloVe-GA，一种用于自然语言表达的多步推理的迭代神经推理网络。在我们的模型中，推理基于带有门控注意力机制的RNN迭代记忆神经网络进行。我们在三个数据集（PARARULES、CONCEPTRULES V1 和 CONCEPTRULES V2）上评估了IMA-GloVe-GA。实验结果表明，带门控注意力的DeepLogic能实现比DeepLogic及其他RNN基线模型更高的测试准确率。当规则被打乱时，我们的模型在分布外泛化方面优于RoBERTa-Large。此外，为解决当前多步推理数据集中推理深度分布不均的问题，我们开发了PARARULE-Plus，这是一个包含更多需要更深推理步骤示例的大型数据集。实验结果表明，加入PARARULE-Plus能提升模型在需要更深推理深度的示例上的性能。源代码和数据可在https://github.com/Strong-AI-Lab/Multi-Step-Deductive-Reasoning-Over-Natural-Language获取。