Large language models (LLMs) are proficient at generating fluent text with minimal task-specific supervision. Yet, their ability to provide well-grounded rationalizations for knowledge-intensive tasks remains under-explored. Such tasks, like commonsense multiple-choice questions, require rationales based on world knowledge to support predictions and refute alternate options. We consider the task of generating knowledge-guided rationalization in natural language by using expert-written examples in a few-shot manner. Surprisingly, crowd-workers preferred knowledge-grounded rationales over crowdsourced rationalizations, citing their factuality, sufficiency, and comprehensive refutations. Although LLMs-generated rationales were preferable, further improvements in conciseness and novelty are required. In another study, we show how rationalization of incorrect model predictions erodes humans' trust in LLM-generated rationales. Motivated by these observations, we create a two-stage pipeline to review task predictions and eliminate potential incorrect decisions before rationalization, enabling trustworthy rationale generation.

翻译：大型语言模型（LLM）能以最少的任务特定监督生成流畅文本。然而，它们为知识密集型任务提供充分依据的理性化能力仍未得到充分探索。这类任务（如常识性多项选择题）需要基于世界知识的推理论证来支持预测并反驳其他选项。我们考虑通过少量专家撰写示例以自然语言生成知识引导的理性化。令人意外的是，众包工作者更偏好知识驱动的推理论证而非众包生成的理性化，认为其具有事实性、充分性和全面的反驳能力。尽管LLM生成的推理论证更受欢迎，但需在简洁性和新颖性方面进一步优化。另一项研究显示，对错误模型预测的理性化会削弱人类对LLM生成推理论证的信任。基于这些观察，我们构建了一个两阶段流水线，在理性化前先审查任务预测并消除潜在错误决策，从而实现可信的推理论证生成。