In a plethora of recent work, large language models (LLMs) demonstrated impressive reasoning ability, but many proposed downstream reasoning tasks focus on performance-wise evaluation. Two fundamental questions persist: 1) how reliable is the quality of reasoning, and 2) can models detect unreliable reasoning? In this paper, we investigate self-contradictory (Self-Contra) reasoning, where the model reasoning does not support predictions. To address 1), we assess the Self-Contra rate across four datasets and delve into finer-grained categories of Self-Contra reasoning. We find that LLMs often contradict themselves when performing reasoning tasks that involve contextual information understanding or commonsense. Importantly, a higher accuracy does not necessarily correspond to a lower Self-Contra rate. The model may appear to generate correct answers but it may take shortcuts in reasoning or skip over contextual evidence, thereby displaying Self-Contra behaviors with compromised reasoning. As for 2), we task GPT-4 with identifying Self-Contra reasoning and finer-grained fallacies. We observe that GPT-4 struggles to effectively detect Self-Contra reasoning, with significantly low performance compared with human judgment. Our results indicate that the current LLMs lack robustness necessary for reliable reasoning and we emphasize the urgent need for establishing best practices in comprehensive reasoning evaluations beyond accuracy-based metrics.

翻译：在近期大量研究中，大语言模型（LLMs）展示了令人印象深刻的推理能力，但许多提出的下游推理任务侧重于性能层面的评估。两个基本问题依然存在：1）推理质量的可靠性如何？2）模型能否检测不可靠的推理？本文中，我们研究自相矛盾推理——即模型推理与预测结果不一致的现象。针对问题1），我们评估了四个数据集上的自相矛盾率，并对自相矛盾推理的细粒度类别进行深入分析。研究发现，LLMs在执行涉及上下文信息理解或常识的推理任务时，经常出现自相矛盾现象。重要的是，更高的准确率并不必然对应更低的自相矛盾率。模型可能看似生成正确答案，但可能走推理捷径或跳过上下文证据，从而表现出推理受损的自相矛盾行为。针对问题2），我们让GPT-4识别自相矛盾推理及其细粒度谬误。我们观察到，GPT-4难以有效检测自相矛盾推理，其表现与人类判断相比显著低下。我们的结果表明，当前LLMs缺乏实现可靠推理所必需的鲁棒性，并强调亟需建立超越准确率指标的全面推理评估最佳实践。