Large language models (LLMs) are proficient at generating fluent text with minimal task-specific supervision. Yet, their ability to provide well-grounded rationalizations for knowledge-intensive tasks remains under-explored. Such tasks, like commonsense multiple-choice questions, require rationales based on world knowledge to support predictions and refute alternate options. We consider the task of generating knowledge-guided rationalization in natural language by using expert-written examples in a few-shot manner. Surprisingly, crowd-workers preferred knowledge-grounded rationales over crowdsourced rationalizations, citing their factuality, sufficiency, and comprehensive refutations. Although LLMs-generated rationales were preferable, further improvements in conciseness and novelty are required. In another study, we show how rationalization of incorrect model predictions erodes humans' trust in LLM-generated rationales. Motivated by these observations, we create a two-stage pipeline to review task predictions and eliminate potential incorrect decisions before rationalization, enabling trustworthy rationale generation.

翻译：大型语言模型（LLMs）能够以最少的任务特定监督生成流畅文本。然而，它们为知识密集型任务提供基于充分依据的推理能力仍待深入探索。此类任务（如常识类多项选择题）需要基于世界知识的推理来支持预测并反驳错误选项。我们探讨了通过少样本方式利用专家撰写的示例生成自然语言知识引导推理的任务。令人惊讶的是，众包工作者更偏好知识驱动的推理而非众包生成的推理，理由在于其事实准确性、充分性及全面的反驳能力。尽管LLMs生成的推理更受青睐，但在简洁性和新颖性方面仍需改进。另一项研究表明，对模型错误预测进行推理会削弱人类对LLMs生成推理的信任。基于这些发现，我们构建了一个两阶段流水线，在推理前先审查任务预测并消除潜在错误决策，从而实现可信赖的推理生成。