Large language models (LLMs) such as ChatGPT have demonstrated superior performance on a variety of natural language processing (NLP) tasks including sentiment analysis, mathematical reasoning and summarization. Furthermore, since these models are instruction-tuned on human conversations to produce "helpful" responses, they can and often will produce explanations along with the response, which we call self-explanations. For example, when analyzing the sentiment of a movie review, the model may output not only the positivity of the sentiment, but also an explanation (e.g., by listing the sentiment-laden words such as "fantastic" and "memorable" in the review). How good are these automatically generated self-explanations? In this paper, we investigate this question on the task of sentiment analysis and for feature attribution explanation, one of the most commonly studied settings in the interpretability literature (for pre-ChatGPT models). Specifically, we study different ways to elicit the self-explanations, evaluate their faithfulness on a set of evaluation metrics, and compare them to traditional explanation methods such as occlusion or LIME saliency maps. Through an extensive set of experiments, we find that ChatGPT's self-explanations perform on par with traditional ones, but are quite different from them according to various agreement metrics, meanwhile being much cheaper to produce (as they are generated along with the prediction). In addition, we identified several interesting characteristics of them, which prompt us to rethink many current model interpretability practices in the era of ChatGPT(-like) LLMs.

翻译：大型语言模型（LLMs）如ChatGPT在情感分析、数学推理和摘要等各类自然语言处理（NLP）任务中展现出卓越性能。此外，由于这些模型经过人类对话的指令微调以生成"有帮助"的回复，它们能够且经常会在回复时附带解释，我们称之为自解释。例如，在分析电影评论的情感时，模型不仅会输出情感倾向性，还会提供解释（如列举评论中"精彩"和"难忘"等情感负载词）。这些自动生成的自解释效果如何？本文针对情感分析任务和特征归因解释这一可解释性文献中最常研究的场景（针对ChatGPT之前的模型）探究此问题。具体而言，我们研究了自解释的多种生成方式，通过一系列评估指标衡量其忠实性，并将其与遮挡法或LIME显著性图等传统解释方法进行比较。通过大量实验，我们发现ChatGPT的自解释性能与传统方法相当，但根据多种一致性指标显示二者存在显著差异，同时生成成本更低（随预测结果同步生成）。此外，我们识别出这些自解释的若干有趣特性，促使我们重新审视ChatGPT式LLM时代当前的诸多模型可解释性实践。