Large language models (LLMs) have demonstrated exceptional performance not only in natural language processing tasks but also in a great variety of non-linguistic domains. In diverse optimization scenarios, there is also a rising trend of applying LLMs. However, whether the application of LLMs in the black-box optimization problems is genuinely beneficial remains unexplored. This paper endeavors to offer deep insights into the potential of LLMs in optimization through a comprehensive investigation, which covers both discrete and continuous optimization problems to assess the efficacy and distinctive characteristics that LLMs bring to this field. Our findings reveal both the limitations and advantages of LLMs in optimization. Specifically, on the one hand, despite the significant power consumed for running the models, LLMs exhibit subpar performance in pure numerical tasks, primarily due to a mismatch between the problem domain and their processing capabilities; on the other hand, although LLMs may not be ideal for traditional numerical optimization, their potential in broader optimization contexts remains promising, where LLMs exhibit the ability to solve problems in non-numerical domains and can leverage heuristics from the prompt to enhance their performance. To the best of our knowledge, this work presents the first systematic evaluation of LLMs for numerical optimization. Our findings pave the way for a deeper understanding of LLMs' role in optimization and guide future application of LLMs in a wide range of scenarios.

翻译：大型语言模型（LLMs）不仅在自然语言处理任务中表现出色，在众多非语言领域也展现出卓越性能。在各类优化场景中，应用LLMs的趋势日益增长。然而，LLMs在黑盒优化问题中的应用是否真正有益仍有待探究。本文通过全面研究，旨在深入揭示LLMs在优化领域的潜力，涵盖离散与连续优化问题，以评估LLMs在该领域的效能与独特特性。我们的研究揭示了LLMs在优化中的局限性与优势。具体而言：一方面，尽管运行模型消耗大量算力，LLMs在纯数值任务中表现欠佳，主要源于问题领域与其处理能力之间的错配；另一方面，尽管LLMs可能不适用于传统数值优化，但它们在更广泛优化场景中的潜力依然可观——LLMs展现出解决非数值领域问题的能力，并能利用提示中的启发式策略提升性能。据我们所知，本研究首次对LLMs在数值优化中的能力进行了系统评估。我们的发现为深入理解LLMs在优化中的作用铺平了道路，并为LLMs在广泛场景中的未来应用提供了指导。