An increasing amount of research in Natural Language Inference (NLI) focuses on the application and evaluation of Large Language Models (LLMs) and their reasoning capabilities. Despite their success, however, LLMs are still prone to factual errors and inconsistencies in their explanations, offering limited control and interpretability for inference in complex domains. In this paper, we focus on ethical NLI, investigating how hybrid neuro-symbolic techniques can enhance the logical validity and alignment of ethical explanations produced by LLMs. Specifically, we present an abductive-deductive framework named Logic-Explainer, which integrates LLMs with an external backward-chaining solver to refine step-wise natural language explanations and jointly verify their correctness, reduce incompleteness and minimise redundancy. An extensive empirical analysis demonstrates that Logic-Explainer can improve explanations generated via in-context learning methods and Chain-of-Thought (CoT) on challenging ethical NLI tasks, while, at the same time, producing formal proofs describing and supporting models' reasoning. As ethical NLI requires commonsense reasoning to identify underlying moral violations, our results suggest the effectiveness of neuro-symbolic methods for multi-step NLI more broadly, opening new opportunities to enhance the logical consistency, reliability, and alignment of LLMs.
翻译:自然语言推理领域的研究日益关注大型语言模型及其推理能力的应用与评估。尽管取得了显著成功,但大型语言模型在解释过程中仍存在事实错误与不一致性,导致在复杂领域推理中缺乏可控性和可解释性。本文聚焦伦理自然语言推理,探究混合神经符号技术如何提升大型语言模型生成的伦理解释的逻辑有效性与对齐性。具体而言,我们提出名为Logic-Explainer的溯因-演绎框架,该框架将大型语言模型与外部反向链求解器相结合,以优化逐步自然语言解释,并联合验证其正确性、减少不完整性及冗余。大量实证分析表明,在具有挑战性的伦理自然语言推理任务上,Logic-Explainer能够改进通过情境学习方法和思维链生成的解释,同时生成描述并支撑模型推理的形式化证明。由于伦理自然语言推理需要常识推理来识别潜在道德违规,我们的研究结果揭示了神经符号方法在更广泛的多步自然语言推理中的有效性,为提升大型语言模型的逻辑一致性、可靠性与对齐性开辟了新路径。