We study the task of conducting structured reasoning as generating a reasoning graph from natural language input using large language models (LLMs). Previous approaches have explored various prompting schemes, yet they suffer from error propagation due to the autoregressive nature and single-pass-based decoding, which lack error correction capability. Additionally, relying solely on a single sample may result in the omission of true nodes and edges. To counter this, we draw inspiration from self-consistency (SC), which involves sampling a diverse set of reasoning chains and taking the majority vote as the final answer. To tackle the substantial challenge of applying SC on generated graphs, we propose MIDGARD (MInimum Description length Guided Aggregation of Reasoning in Directed acyclic graph) that leverages Minimum Description Length (MDL)-based formulation to identify consistent properties among the different graph samples generated by an LLM. This formulation helps reject properties that appear in only a few samples, which are likely to be erroneous, while enabling the inclusion of missing elements without compromising precision. Our method demonstrates superior performance than comparisons across various structured reasoning tasks, including argument structure extraction, explanation graph generation, inferring dependency relations among actions for everyday tasks, and semantic graph generation from natural texts.

翻译：我们研究将结构化推理任务定义为使用大语言模型从自然语言输入生成推理图。先前的方法已探索了多种提示方案，但由于自回归特性和基于单次解码的机制缺乏纠错能力，它们容易受到误差传播的影响。此外，仅依赖单个样本可能导致真实节点和边的遗漏。针对此问题，我们受到自洽性方法的启发，该方法通过采样一组多样化的推理链并以多数投票确定最终答案。为应对将自洽性应用于生成图这一重大挑战，我们提出了MIDGARD（基于最小描述长度的有向无环图推理聚合方法），该方法利用基于最小描述长度的形式化框架，来识别由大语言模型生成的不同图样本间的一致属性。该框架有助于剔除仅出现在少数样本中（很可能错误的）的属性，同时能够在保证精度的前提下纳入缺失元素。我们的方法在多种结构化推理任务中均表现出优于对比模型的性能，这些任务包括论元结构抽取、解释图生成、日常任务中动作间依赖关系推理，以及从自然文本生成语义图。