Large Language Models (LLMs) have shown remarkable reasoning performance but struggle with multi-step deductive reasoning involving a series of rule application steps, especially when rules are presented non-sequentially. Our preliminary analysis shows that while LLMs excel in single-step rule application, their performance drops significantly in multi-step scenarios due to the challenge in rule grounding. It requires anchoring the applicable rule and supporting facts at each step, amidst multiple input rules, facts, and inferred facts. To address this, we propose augmenting LLMs with external working memory and introduce a neurosymbolic framework for rule application. The memory stores facts and rules in both natural language and symbolic forms, enabling precise tracking. Utilizing this memory, our framework iteratively performs symbolic rule grounding and LLM-based rule implementation. The former matches predicates and variables of symbolic rules and facts to ground applicable rules at each step. Experiments indicate our framework's effectiveness in rule application and its robustness across various steps and settings~\footnote{Code and data are available at \url{https://github.com/SiyuanWangw/RuleApplication}.}.

翻译：大型语言模型（LLMs）已展现出卓越的推理性能，但在涉及一系列规则应用步骤的多步演绎推理任务中表现欠佳，尤其是当规则非顺序呈现时。我们的初步分析表明，尽管LLMs在单步规则应用上表现出色，但在多步场景中，由于规则落地（rule grounding）的挑战，其性能显著下降。这需要在多个输入规则、事实及推断事实中，于每一步锚定适用的规则及其支持事实。为解决此问题，我们提出通过外部工作记忆增强LLMs，并引入一个用于规则应用的神经符号框架。该记忆以自然语言和符号形式存储事实与规则，从而实现精确追踪。利用此记忆，我们的框架迭代执行符号规则落地和基于LLM的规则实现。前者通过匹配符号规则与事实的谓词和变量，在每一步落地适用的规则。实验表明，我们的框架在规则应用上具有有效性，并在不同步骤和设置下展现出稳健性~\footnote{代码与数据可在 \url{https://github.com/SiyuanWangw/RuleApplication} 获取。}。