Large Language Models (LLMs) can achieve strong performance on many tasks by producing step-by-step reasoning before giving a final output, often referred to as chain-of-thought reasoning (CoT). It is tempting to interpret these CoT explanations as the LLM's process for solving a task. However, we find that CoT explanations can systematically misrepresent the true reason for a model's prediction. We demonstrate that CoT explanations can be heavily influenced by adding biasing features to model inputs -- e.g., by reordering the multiple-choice options in a few-shot prompt to make the answer always "(A)" -- which models systematically fail to mention in their explanations. When we bias models toward incorrect answers, they frequently generate CoT explanations supporting those answers. This causes accuracy to drop by as much as 36% on a suite of 13 tasks from BIG-Bench Hard, when testing with GPT-3.5 from OpenAI and Claude 1.0 from Anthropic. On a social-bias task, model explanations justify giving answers in line with stereotypes without mentioning the influence of these social biases. Our findings indicate that CoT explanations can be plausible yet misleading, which risks increasing our trust in LLMs without guaranteeing their safety. CoT is promising for explainability, but our results highlight the need for targeted efforts to evaluate and improve explanation faithfulness.

翻译：大型语言模型（LLMs）通过生成逐步推理后再输出最终答案（即思维链推理，CoT），在众多任务中展现出强大性能。人们倾向于将这些CoT解释视为LLM解决任务的推理过程。然而，我们发现CoT解释可能系统性地歪曲模型预测的真实原因。我们证明，在模型输入中添加偏差特征（例如，在少样本提示中重新排列多选题选项，使答案始终为"(A)"）会严重影响CoT解释，而模型在解释中却系统性地未提及这些特征。当我们将模型导向错误答案时，它们经常生成支持这些答案的CoT解释。在BIG-Bench Hard套件的13个任务中，使用OpenAI的GPT-3.5和Anthropic的Claude 1.0进行测试时，准确率因此下降了高达36%。在社交偏见任务中，模型解释证明其给出符合刻板印象的答案合理，却未提及这些社会偏见的影响。我们的研究结果表明，CoT解释可能看似合理却具有误导性，这增加了我们对LLMs的信任风险，却未保障其安全性。CoT在可解释性方面具有潜力，但我们的研究结果强调了需要开展针对性工作来评估和改进解释的忠实性。