Large Language Models (LLMs) such as GPT-4 have demonstrated their ability to understand natural language and generate complex code snippets. This paper introduces a novel Large Language Model Evolutionary Algorithm (LLaMEA) framework, leveraging GPT models for the automated generation and refinement of algorithms. Given a set of criteria and a task definition (the search space), LLaMEA iteratively generates, mutates and selects algorithms based on performance metrics and feedback from runtime evaluations. This framework offers a unique approach to generating optimized algorithms without requiring extensive prior expertise. We show how this framework can be used to generate novel black-box metaheuristic optimization algorithms automatically. LLaMEA generates multiple algorithms that outperform state-of-the-art optimization algorithms (Covariance Matrix Adaptation Evolution Strategy and Differential Evolution) on the five dimensional black box optimization benchmark (BBOB). The algorithms also show competitive performance on the 10- and 20-dimensional instances of the test functions, although they have not seen such instances during the automated generation process. The results demonstrate the feasibility of the framework and identify future directions for automated generation and optimization of algorithms via LLMs.

翻译：诸如GPT-4等大语言模型（LLMs）已展现出理解自然语言和生成复杂代码片段的能力。本文提出了一种新颖的大语言模型进化算法（LLaMEA）框架，该框架利用GPT模型实现算法的自动生成与优化。给定一组标准和一个任务定义（即搜索空间），LLaMEA基于性能指标和运行时评估的反馈，迭代地生成、变异和选择算法。该框架提供了一种无需大量先验专业知识即可生成优化算法的独特方法。我们展示了如何使用该框架自动生成新颖的黑盒元启发式优化算法。在五维黑盒优化基准测试（BBOB）上，LLaMEA生成的多种算法性能优于最先进的优化算法（协方差矩阵自适应进化策略和差分进化）。这些算法在测试函数的10维和20维实例上也表现出有竞争力的性能，尽管它们在自动生成过程中并未见过此类实例。结果证明了该框架的可行性，并为通过大语言模型实现算法的自动生成与优化指明了未来研究方向。