Despite advancements in LLMs, knowledge-based reasoning remains a longstanding issue due to the fragility of knowledge recall and inference. Existing methods primarily encourage LLMs to autonomously plan and solve problems or to extensively sample reasoning chains without addressing the conceptual and inferential fallacies. Attempting to alleviate inferential fallacies and drawing inspiration from multi-agent collaboration, we present a framework to increase faithfulness and causality for knowledge-based reasoning. Specifically, we propose to employ multiple intelligent agents (i.e., reasoner and causal evaluator) to work collaboratively in a reasoning-and-consensus paradigm for elevated reasoning faithfulness. The reasoners focus on providing solutions with human-like causality to solve open-domain problems. On the other hand, the causal evaluator agent scrutinizes if the answer in a solution is causally deducible from the question and vice versa, with a counterfactual answer replacing the original. According to the extensive and comprehensive evaluations on a variety of knowledge reasoning tasks (e.g., science question answering and commonsense reasoning), our framework outperforms all compared state-of-the-art approaches by large margins.

翻译：尽管大语言模型取得了进展，但由于知识检索与推理的脆弱性，基于知识的推理仍是一个长期难题。现有方法主要鼓励大语言模型自主规划与解决问题，或对推理链进行大量采样，但未能解决概念性与推理谬误。为缓解推理谬误并受多智能体协作启发，我们提出一个框架以增强基于知识推理的可靠性与因果性。具体而言，我们提议采用多个智能体（即推理者与因果评估者）在“推理-共识”范式中协同工作，以提升推理可靠性。推理者专注于提供具有类人因果性的解决方案，以解决开放领域问题。另一方面，因果评估者智能体审视解答中的答案是否可从问题中因果推导而来，反之亦然，并通过反事实答案替换原始答案。在多种知识推理任务（例如科学问答与常识推理）上的广泛全面评估表明，我们的框架以显著优势超越所有对比的现有最优方法。