Numerous recent works aim to enhance the efficacy of Large Language Models (LLMs) through strategic prompting. In particular, the Optimization by PROmpting (OPRO) approach provides state-of-the-art performance by leveraging LLMs as optimizers where the optimization task is to find instructions that maximize the task accuracy. In this paper, we revisit OPRO for automated prompting with relatively small-scale LLMs, such as LLaMa-2 family and Mistral 7B. Our investigation reveals that OPRO shows limited effectiveness in small-scale LLMs, with limited inference capabilities constraining optimization ability. We suggest future automatic prompting engineering to consider both model capabilities and computational costs. Additionally, for small-scale LLMs, we recommend direct instructions that clearly outline objectives and methodologies as robust prompt baselines, ensuring efficient and effective prompt engineering in ongoing research.

翻译：近期众多研究致力于通过策略性提示增强大语言模型（LLM）的效能。其中，基于提示的优化（OPRO）方法通过将LLM作为优化器实现了最先进的性能，其优化任务是寻找能最大化任务准确率的指令。本文重新审视了使用相对小规模LLM（如LLaMa-2系列和Mistral 7B）进行自动化提示的OPRO方法。研究发现，OPRO在小规模LLM中效果有限，其受限的推理能力制约了优化性能。我们建议未来的自动化提示工程应同时考虑模型能力与计算成本。此外，针对小规模LLM，我们推荐采用明确阐述目标与方法论的直接指令作为稳健的提示基准，以确保持续研究中的提示工程既高效又有效。