Large Foundational Language Models are capable of performing many tasks at a high level but are difficult to deploy in many applications because of their size and proprietary ownership. Many will be motivated to distill specific capabilities of foundational models into smaller models that can be owned and controlled. In the development of a therapeutic chatbot, we wish to distill a capability known as reflective listening, in which a therapist produces reflections of client speech. These reflections either restate what a client has said, or connect what was said to a relevant observation, idea or guess that encourages and guides the client to continue contemplation. In this paper, we present a method for distilling the generation of reflections from a Foundational Language Model (GPT-4) into smaller models. We first show that GPT-4, using zero-shot prompting, can generate reflections at near 100% success rate, superior to all previous methods. Using reflections generated by GPT-4, we fine-tune different sizes of the GPT-2 family. The GPT-2-small model achieves 83% success on a hold-out test set and the GPT-2 XL achieves 90% success. We also show that GPT-4 can help in the labor-intensive task of evaluating the quality of the distilled models, using it as a zero-shot classifier. Using triple-human review as a guide, the classifier achieves a Cohen-Kappa of 0.66, a substantial inter-rater reliability figure.

翻译：大型基础语言模型能够高水平执行多项任务，但由于其规模庞大且为专有所有权，难以在众多应用中部署。许多人将致力于将基础模型的特定能力蒸馏到更小、可拥有与可控的模型中。在开发治疗性聊天机器人时，我们希望蒸馏一种被称为"反思性倾听"的能力，即治疗师对来访者话语产生反思。这些反思要么重述来访者所述内容，要么将所述内容与相关观察、想法或猜测联系起来，从而鼓励并引导来访者继续思考。本文提出一种方法，将反思生成能力从基础语言模型（GPT-4）蒸馏到更小模型中。我们首先证明，GPT-4通过零样本提示能够以近乎100%的成功率生成反思，优于以往所有方法。利用GPT-4生成的反思，我们对不同大小的GPT-2系列模型进行微调。其中GPT-2-small模型在保留测试集上达到83%的成功率，而GPT-2 XL达到90%的成功率。我们还证明，GPT-4可协助完成评估蒸馏模型质量这一劳动密集型任务，将其作为零样本分类器使用。以三人人工评审为基准，该分类器实现了0.66的Cohen-Kappa系数，达到显著的评分者间信度水平。