Chain of thought prompting successfully improves the reasoning capabilities of large language models, achieving state of the art results on a range of datasets. However, these reasoning capabilities only appear to emerge in models with a size of over 100 billion parameters. In this paper, we explore the transfer of such reasoning capabilities to models with less than 100 billion parameters via knowledge distillation. Specifically, we finetune a student model on the chain of thought outputs generated by a larger teacher model. Our experiments show that the proposed method improves task performance across arithmetic, commonsense and symbolic reasoning datasets. For example, the accuracy of T5 XXL on GSM8K improves from 8.11% to 21.99% when finetuned on PaLM-540B generated chains of thought.

翻译：链式思考提示成功提升了大语言模型的推理能力，在一系列数据集上取得了最优结果。然而，这些推理能力似乎仅出现在参数规模超过1000亿的模型中。本文探讨通过知识蒸馏将此类推理能力迁移至参数规模低于1000亿的模型。具体而言，我们利用更大教师模型生成的链式思考输出对学生模型进行微调。实验表明，所提出的方法在算术、常识和符号推理数据集上均能提升任务性能。例如，当T5 XXL在PaLM-540B生成的链式思考上微调后，其在GSM8K上的准确率从8.11%提升至21.99%。