The remarkable performance of pre-trained large language models has revolutionised various natural language processing applications. Due to huge parametersizes and extensive running costs, companies or organisations tend to transfer the models to the target task by zero-shot prompting techniques. However, the prohibitive costs of tokens and time have hindered their adoption in applications. We propose OverPrompt, leveraging the in-context learning capability of LLMs to handle multiple task inputs, thereby reducing token and time costs. This approach could potentially improve task performance during API queries due to better conditional distribution mapping. Evaluated across diverse classification datasets, our experiments show that OverPrompt can achieve cost-efficient zero-shot classification without causing significant detriment to task performance, and in some cases, even improving it. An ablation study conducted on various LLMs, along with an investigation into the robustness of our prompting strategy to different input ordering, offers valuable insights into the broader applicability of our method across diverse tasks. These findings also suggest a more seamless integration of our method with LLMs through an API.

翻译：预训练大型语言模型的卓越性能已彻底改变了自然语言处理领域的多种应用。由于模型参数规模庞大且运行成本高昂，企业或组织倾向于通过零样本提示技术将模型迁移至目标任务。然而，提示所需的令牌数量与时间成本过高，阻碍了其在应用中的广泛采用。我们提出OverPrompt方法，利用大型语言模型的上下文学习能力处理多个任务输入，从而降低令牌与时间成本。该方法通过优化条件分布映射，在API查询期间可潜在提升任务性能。在多个分类数据集上的评估表明，OverPrompt能够在不显著损害任务性能的前提下实现高效零样本分类，甚至在部分场景中提升了性能。针对不同大型语言模型的消融实验，以及对我们提示策略在输入顺序鲁棒性方面的探究，为该方法在多种任务中的广泛适用性提供了重要见解。这些发现还表明，我们的方法可通过API与大型语言模型实现更无缝的集成。