Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort. We propose a simple and nonparametric solution to this problem, Automatic Prompt Optimization (APO), which is inspired by numerical gradient descent to automatically improve prompts, assuming access to training data and an LLM API. The algorithm uses minibatches of data to form natural language ``gradients'' that criticize the current prompt. The gradients are then ``propagated'' into the prompt by editing the prompt in the opposite semantic direction of the gradient. These gradient descent steps are guided by a beam search and bandit selection procedure which significantly improves algorithmic efficiency. Preliminary results across three benchmark NLP tasks and the novel problem of LLM jailbreak detection suggest that Automatic Prompt Optimization can outperform prior prompt editing techniques and improve an initial prompt's performance by up to 31\%, by using data to rewrite vague task descriptions into more precise annotation instructions.

翻译：大语言模型（LLMs）作为通用智能体展现出卓越性能，但其能力高度依赖于通过繁琐试错法手工编写的提示词。针对该问题，我们提出一种简单且非参数的解决方案——自动提示词优化（APO），该方法受数值梯度下降启发，可在访问训练数据和大语言模型API的条件下自动优化提示词。算法通过数据小批量形成批评当前提示词的自然语言"梯度"，随后沿梯度的语义相反方向编辑提示词，实现梯度的"反向传播"。该梯度下降过程由束搜索与赌臂选择机制引导，显著提升了算法效率。在三项基准自然语言处理任务及大语言模型越狱检测新问题上的初步实验表明，自动提示词优化可超越现有提示词编辑技术，通过利用数据将模糊任务描述重写为精确标注指令，使初始提示词性能提升高达31%。