Combining deep learning with symbolic logic reasoning aims to capitalize on the success of both fields and is drawing increasing attention. Inspired by DeepLogic, an end-to-end model trained to perform inference on logic programs, we introduce IMA-GloVe-GA, an iterative neural inference network for multi-step reasoning expressed in natural language. In our model, reasoning is performed using an iterative memory neural network based on RNN with a gate attention mechanism. We evaluate IMA-GloVe-GA on three datasets: PARARULES, CONCEPTRULES V1 and CONCEPTRULES V2. Experimental results show DeepLogic with gate attention can achieve higher test accuracy than DeepLogic and other RNN baseline models. Our model achieves better out-of-distribution generalisation than RoBERTa-Large when the rules have been shuffled. Furthermore, to address the issue of unbalanced distribution of reasoning depths in the current multi-step reasoning datasets, we develop PARARULE-Plus, a large dataset with more examples that require deeper reasoning steps. Experimental results show that the addition of PARARULE-Plus can increase the model's performance on examples requiring deeper reasoning depths. The source code and data are available at https://github.com/Strong-AI-Lab/Multi-Step-Deductive-Reasoning-Over-Natural-Language.

翻译：将深度学习与符号逻辑推理相结合，旨在利用两个领域的成功并日益受到关注。受DeepLogic（一种端到端模型，经过训练可对逻辑程序进行推理）的启发，我们引入了IMA-GloVe-GA，一个用于自然语言中表达的多步推理的迭代神经推理网络。在我们的模型中，推理是通过基于带门控注意力机制的RNN的迭代记忆神经网络来执行的。我们在三个数据集上评估了IMA-GloVe-GA：PARARULES、CONCEPTRULES V1和CONCEPTRULES V2。实验结果表明，带门控注意力的DeepLogic比DeepLogic和其他RNN基线模型能达到更高的测试准确率。当规则被打乱时，我们的模型相比RoBERTa-Large实现了更好的分布外泛化能力。此外，为了解决当前多步推理数据集中推理深度分布不平衡的问题，我们开发了PARARULE-Plus，这是一个包含更多需要更深推理步骤示例的大规模数据集。实验结果表明，PARARULE-Plus的加入可以增强模型在需要更深推理深度的示例上的性能。源码和数据可在https://github.com/Strong-AI-Lab/Multi-Step-Deductive-Reasoning-Over-Natural-Language获取。