A well-engineered prompt can increase the performance of large language models; automatic prompt optimization techniques aim to increase performance without requiring human effort to tune the prompts. One leading class of prompt optimization techniques introduces the analogy of textual gradients. We investigate the behavior of these textual gradient methods through a series of experiments and case studies. While such methods often result in a performance improvement, our experiments suggest that the gradient analogy does not accurately explain their behavior. Our insights may inform the selection of prompt optimization strategies, and development of new approaches.

翻译：精心设计的提示能够提升大语言模型的性能；自动提示优化技术旨在无需人工调优提示的情况下提高性能。一类主流的提示优化技术引入了文本梯度的类比。我们通过一系列实验和案例研究探讨了这些文本梯度方法的行为。尽管此类方法常能带来性能提升，但我们的实验表明梯度类比并不能准确解释其行为。我们的见解可为提示优化策略的选择及新方法的开发提供参考。