Large language models (LLMs) perform well at a myriad of tasks, but explaining the processes behind this performance is a challenge. This paper investigates whether LLMs can give faithful high-level explanations of their own internal processes. To explore this, we introduce a dataset, ArticulateRules, of few-shot text-based classification tasks generated by simple rules. Each rule is associated with a simple natural-language explanation. We test whether models that have learned to classify inputs competently (both in- and out-of-distribution) are able to articulate freeform natural language explanations that match their classification behavior. Our dataset can be used for both in-context and finetuning evaluations. We evaluate a range of LLMs, demonstrating that articulation accuracy varies considerably between models, with a particularly sharp increase from GPT-3 to GPT-4. We then investigate whether we can improve GPT-3's articulation accuracy through a range of methods. GPT-3 completely fails to articulate 7/10 rules in our test, even after additional finetuning on correct explanations. We release our dataset, ArticulateRules, which can be used to test self-explanation for LLMs trained either in-context or by finetuning.

翻译：大型语言模型（LLMs）在众多任务中表现出色，但解释其性能背后的过程仍是一项挑战。本文探究LLMs能否对其内部处理过程提供忠实的高层次解释。为此，我们引入一个名为ArticulateRules的数据集，该数据集由基于简单规则生成的少样本文本分类任务构成，每条规则均附有简洁的自然语言解释。我们检验那些已学会对输入（包括分布内和分布外输入）进行熟练分类的模型，能否生成与其分类行为匹配的自由形式自然语言解释。该数据集可同时用于上下文学习和微调评估。我们对多种LLMs进行了评估，结果表明各模型在阐述准确性上存在显著差异，尤其从GPT-3到GPT-4出现了急剧提升。随后，我们探索了通过多种方法提升GPT-3阐述准确性的可能性——即使在对正确解释进行额外微调后，GPT-3在测试的10条规则中仍有7条完全无法阐述。我们公开了ArticulateRules数据集，该数据集可用于测试通过上下文学习或微调训练的LLMs的自我解释能力。