Large language models (LLMs) enable system builders today to create competent NLP systems through prompting, where they only need to describe the task in natural language and provide a few examples. However, in other ways, LLMs are a step backward from traditional special-purpose NLP models; they require extensive computational resources for deployment and can be gated behind APIs. In this paper, we propose Prompt2Model, a general-purpose method that takes a natural language task description like the prompts provided to LLMs, and uses it to train a special-purpose model that is conducive to deployment. This is done through a multi-step process of retrieval of existing datasets and pretrained models, dataset generation using LLMs, and supervised fine-tuning on these retrieved and generated datasets. Over three tasks, we demonstrate that given the same few-shot prompt as input, Prompt2Model trains models that outperform the results of a strong LLM, gpt-3.5-turbo, by an average of 20% while being up to 700 times smaller. We also show that this data can be used to obtain reliable performance estimates of model performance, enabling model developers to assess model reliability before deployment. Prompt2Model is available open-source at https://github.com/neulab/prompt2model.

翻译：大型语言模型（LLMs）使系统构建者如今能够通过提示（prompting）创建胜任的自然语言处理（NLP）系统，只需用自然语言描述任务并提供少量示例即可。然而，从其他方面看，LLMs相比传统的专用NLP模型是一种倒退；它们需要大量计算资源进行部署，且可能受限于API接口。在本文中，我们提出Prompt2Model——一种通用方法，它接收类似提供给LLMs的自然语言任务描述（prompt），并利用该描述训练一个易于部署的专用模型。这一过程通过多步骤完成：检索现有数据集和预训练模型、使用LLMs生成数据集，以及对检索和生成的数据集进行监督微调。在三个任务上，我们证明，给定相同的少样本提示（few-shot prompt）作为输入，Prompt2Model训练的模型相比强LLM模型gpt-3.5-turbo，性能平均提升20%，而模型规模却小至其1/700。我们还展示，这些数据可用于获得可靠的模型性能估计，使模型开发者在部署前评估模型可靠性。Prompt2Model以开源形式发布于https://github.com/neulab/prompt2model。