Large Language Models (LLMs) can perform various natural language processing tasks with suitable instruction prompts. However, designing effective prompts manually is challenging and time-consuming. Existing methods for automatic prompt optimization either lack flexibility or efficiency. In this paper, we propose an effective approach to automatically select the optimal prompt for a given input from a finite set of synthetic candidate prompts. Our approach consists of three steps: (1) clustering the training data and generating candidate prompts for each cluster using an LLM-based prompt generator; (2) synthesizing a dataset of input-prompt-output tuples for training a prompt evaluator to rank the prompts based on their relevance to the input; (3) using the prompt evaluator to select the best prompt for a new input at test time. Our approach balances prompt generality-specificity and eliminates the need for resource-intensive training and inference. It demonstrates competitive performance on zero-shot question-answering datasets: GSM8K, MultiArith, and AQuA.

翻译：大型语言模型（LLMs）能够通过合适的指令提示执行多种自然语言处理任务。然而，手动设计有效的提示既具挑战性又耗时。现有的自动提示优化方法要么缺乏灵活性，要么效率低下。本文提出了一种有效方法，能够从有限的人工合成候选提示集合中自动为给定输入选择最优提示。该方法包含三个步骤：（1）对训练数据进行聚类，并利用基于LLM的提示生成器为每个聚类生成候选提示；（2）合成输入-提示-输出三元组数据集，用于训练提示评估器，根据提示与输入的相关性对其进行排序；（3）在测试阶段，使用提示评估器为新输入选择最佳提示。该方法平衡了提示的通用性与特异性，并消除了对资源密集型训练和推理的需求。在零样本问答数据集GSM8K、MultiArith和AQuA上，该方法展示了具有竞争力的性能。