Exploiting large language models (LLMs) to tackle deductive reasoning has garnered growing attention. It still remains highly challenging to achieve satisfactory results in complex deductive problems, characterized by plenty of premises (i.e., facts or rules) entailing intricate relationships among entities and requiring multi-hop reasoning. One intuitive solution is to decompose the original task into smaller sub-tasks, and then chain the multiple casual reasoning steps together in a forward (e.g., Selection-Inference) or backward (e.g., LAMBADA) direction. However, these techniques inevitably necessitate a large number of overall stages, leading to computationally expensive operations and a higher possibility of making misleading steps. In addition to stage-by-stage decomposition, we draw inspiration from another aspect of human problem-solving. Humans tend to distill the most relevant information and organize their thoughts systematically (e.g., creating mind maps), which assists them in answering questions or drawing conclusions precisely and quickly. In light of this, we propose a novel reasoning approach named Concise and Organized Perception (COP). COP carefully analyzes the given statements to efficiently identify the most pertinent information while eliminating redundancy. It then prompts the LLMs in a more organized form that adapts to the model's inference process. By perceiving concise and organized proofs, the deductive reasoning abilities of LLMs can be better elicited, and the risk of acquiring errors caused by excessive reasoning stages is mitigated. Furthermore, our approach can be combined with the aforementioned ones to further boost their performance. Extensive experimental results on three popular deductive benchmarks (i.e., ProofWriter, PrOntoQA and PrOntoQA-OOD) show that COP significantly outperforms previous state-of-the-art methods.
翻译:利用大型语言模型(LLMs)处理演绎推理已受到越来越多的关注。然而,在复杂的演绎问题中,由于存在大量前提(即事实或规则),这些前提蕴含实体间错综复杂的关系并需要多跳推理,因此要获得令人满意的结果仍极具挑战性。一种直观的解决方案是将原始任务分解为较小的子任务,然后通过前向(如Selection-Inference)或后向(如LAMBADA)方向将多个因果推理步骤串联起来。然而,这些技术不可避免地需要大量总体阶段,从而导致计算开销高昂,并增加做出误导性步骤的可能性。除了逐步分解之外,我们还从人类问题解决的另一个方面获得启发。人类倾向于提炼最相关的信息并系统地组织其思路(例如创建思维导图),这有助于他们精确且快速地回答问题或得出结论。基于此,我们提出了一种新颖的推理方法,名为简洁有序感知(COP)。COP仔细分析给定陈述,高效识别最相关信息,同时消除冗余。然后,它以更有序的形式提示LLMs,使其适应模型的推理过程。通过感知简洁有序的证据,LLMs的演绎推理能力得以更好地激发,并降低了因过多推理阶段而导致错误的风险。此外,我们的方法可与前述方法结合,以进一步提升其性能。在三个流行的演绎推理基准(即ProofWriter、PrOntoQA和PrOntoQA-OOD)上的大量实验结果表明,COP显著优于以往最先进的方法。