Large Language Models (LLMs) have shown potential in reasoning over structured environments, e.g., knowledge graph and table. Such tasks typically require multi-hop reasoning, i.e., match natural language utterance with instances in the environment. Previous methods leverage LLMs to incrementally build a reasoning path, where the LLMs either invoke tools or pick up schemas by step-by-step interacting with the environment. We propose Reasoning-Path-Editing (Readi), a novel framework where LLMs can efficiently and faithfully reason over structured environments. In Readi, LLMs initially generate a reasoning path given a query, and edit the path only when necessary. We instantiate the path on structured environments and provide feedback to edit the path if anything goes wrong. Experimental results on three KGQA and two TableQA datasets show the effectiveness of Readi, significantly surpassing previous LLM-based methods (by 9.1% Hit@1 on WebQSP, 12.4% on MQA-3H and 9.5% on WTQ), comparable with state-of-the-art fine-tuned methods (67% on CWQ and 74.7% on WebQSP) and substantially boosting the vanilla LLMs (by 14.9% on CWQ). Our code will be available on https://aka.ms/readi.

翻译：大语言模型（LLMs）在结构化环境（例如知识图谱和表格）上的推理已展现出潜力。此类任务通常需要多跳推理，即将自然语言表述与环境中的实例进行匹配。先前的方法利用LLMs逐步构建推理路径，在此过程中，LLMs要么调用工具，要么通过与环境的逐步交互来获取模式。我们提出了推理路径编辑（Readi），这是一个新颖的框架，使得LLMs能够高效且忠实地在结构化环境上进行推理。在Readi中，LLMs首先根据查询生成一个初始推理路径，仅在必要时才对该路径进行编辑。我们在结构化环境上实例化该路径，并在出现问题时提供反馈以编辑路径。在三个KGQA和两个TableQA数据集上的实验结果表明了Readi的有效性，其性能显著超越了先前基于LLM的方法（在WebQSP上Hit@1提升9.1%，在MQA-3H上提升12.4%，在WTQ上提升9.5%），与最先进的微调方法相当（在CWQ上达到67%，在WebQSP上达到74.7%），并大幅提升了原始LLMs的性能（在CWQ上提升14.9%）。我们的代码将在 https://aka.ms/readi 上提供。