Deploying large language models (LLMs) is challenging because they are memory inefficient and compute-intensive for practical applications. In reaction, researchers train smaller task-specific models by either finetuning with human labels or distilling using LLM-generated labels. However, finetuning and distillation require large amounts of training data to achieve comparable performance to LLMs. We introduce Distilling step-by-step, a new mechanism that (a) trains smaller models that outperform LLMs, and (b) achieves so by leveraging less training data needed by finetuning or distillation. Our method extracts LLM rationales as additional supervision for training small models within a multi-task framework. We present three findings across 4 NLP benchmarks: First, compared to both finetuning and distillation, our mechanism achieves better performance with much fewer labeled/unlabeled training examples. Second, compared to few-shot prompted LLMs, we achieve better performance using substantially smaller model sizes. Third, we reduce both the model size and the amount of data required to outperform LLMs; our finetuned 770M T5 model outperforms the few-shot prompted 540B PaLM model using only 80% of available data on a benchmark, whereas standard finetuning the same T5 model struggles to match even by using 100% of the dataset. We release the code at: https://github.com/google-research/distilling-step-by-step .

翻译：部署大型语言模型（LLMs）在实际应用中因内存效率低下和计算密集而具有挑战性。为此，研究者通过两种方式训练小型任务专用模型：基于人工标注的微调，或利用LLM生成的标签进行蒸馏。然而，微调和蒸馏都需要大量训练数据才能达到与LLMs相当的性能。我们提出"逐层蒸馏"这一新机制，它能（a）训练出超越LLMs的小型模型，且（b）在达成该目标时使用的训练数据少于微调或蒸馏所需。该方法将LLM的推理过程作为额外监督信号，在多任务框架中训练小型模型。我们在4项自然语言处理基准测试中得出三个发现：第一，与微调和蒸馏相比，我们的机制用更少的标注/未标注训练样本即实现更优性能；第二，与少样本提示的LLMs相比，我们用显著更小的模型规模获得更好表现；第三，我们同时缩减了超越LLMs所需的模型规模和训练数据量——在某个基准测试中，我们微调后的770M参数T5模型仅使用80%可用数据即超越少样本提示的540B参数PaLM模型，而相同T5模型的标准微调即便使用100%数据集仍难以匹敌。我们已在https://github.com/google-research/distilling-step-by-step 开源代码。