We study the task of conducting structured reasoning as generating a reasoning graph from natural language input using large language models (LLMs). Previous approaches have explored various prompting schemes, yet they suffer from error propagation due to the autoregressive nature and single-pass-based decoding, which lack error correction capability. Additionally, relying solely on a single sample may result in the omission of true nodes and edges. To counter this, we draw inspiration from self-consistency (SC), which involves sampling a diverse set of reasoning chains and taking the majority vote as the final answer. To tackle the substantial challenge of applying SC on generated graphs, we propose MIDGARD (MInimum Description length Guided Aggregation of Reasoning in Directed acyclic graph) that leverages Minimum Description Length (MDL)-based formulation to identify consistent properties among the different graph samples generated by an LLM. This formulation helps reject properties that appear in only a few samples, which are likely to be erroneous, while enabling the inclusion of missing elements without compromising precision. Our method demonstrates superior performance than comparisons across various structured reasoning tasks, including argument structure extraction, explanation graph generation, inferring dependency relations among actions for everyday tasks, and semantic graph generation from natural texts.

翻译：我们研究如何利用大型语言模型从自然语言输入生成推理图以进行结构化推理的任务。先前的方法探索了多种提示策略，但由于自回归特性和基于单次生成的解码方式，这些方法存在错误传播且缺乏纠错能力。此外，仅依赖单一采样结果可能导致真实节点和边的遗漏。为解决这一问题，我们从自一致性方法中获得灵感，该方法通过采样多样化的推理链并取多数投票结果作为最终答案。为应对在生成图上应用自一致性的重大挑战，我们提出MIDGARD（基于最小描述长度的有向无环图推理聚合方法），该方法利用最小描述长度框架识别大语言模型生成的不同图样本间的一致特性。该公式可拒绝仅出现在少数样本中（可能包含错误）的特性，同时在不牺牲精确性的前提下纳入缺失元素。我们的方法在各类结构化推理任务（包括论证结构提取、解释图生成、日常任务动作依赖关系推断及自然文本语义图生成）中均展现出优于对比方法的性能。