We study the potential of using large language models (LLMs) as an interactive optimizer for solving maximization problems in a text space using natural language and numerical feedback. Inspired by the classical optimization literature, we classify the natural language feedback into directional and non-directional, where the former is a generalization of the first-order feedback to the natural language space. We find that LLMs are especially capable of optimization when they are provided with {directional feedback}. Based on this insight, we design a new LLM-based optimizer that synthesizes directional feedback from the historical optimization trace to achieve reliable improvement over iterations. Empirically, we show our LLM-based optimizer is more stable and efficient in solving optimization problems, from maximizing mathematical functions to optimizing prompts for writing poems, compared with existing techniques.

翻译：本研究探讨了利用大语言模型（LLMs）作为交互式优化器，通过自然语言和数值反馈解决文本空间最大化问题的潜力。受经典优化文献启发，我们将自然语言反馈分为方向性和非方向性两类，其中前者是一阶反馈在自然语言空间中的推广。我们发现，当大语言模型获得{方向性反馈}时，其优化能力尤为突出。基于这一洞见，我们设计了一种新的大语言模型优化器，该优化器能够从历史优化轨迹中综合出方向性反馈，从而在迭代过程中实现可靠的性能提升。实验表明，与现有技术相比，我们的大语言模型优化器在解决优化问题（从数学函数最大化到诗歌创作提示优化）时，表现更为稳定和高效。