The omnipresence of NP-hard combinatorial optimization problems (COPs) compels domain experts to engage in trial-and-error heuristic design process. The long-standing endeavor of design automation has gained new momentum with the rise of large language models (LLMs). This paper introduces Language Hyper-Heuristics (LHHs), an emerging variant of Hyper-Heuristics that leverages LLMs for heuristic generation, featuring minimal manual intervention and open-ended heuristic spaces. To empower LHHs, we present Reflective Evolution (ReEvo), a generic searching framework that emulates the reflective design approach of human experts while far surpassing human capabilities with its scalable LLM inference, Internet-scale domain knowledge, and powerful evolutionary search. Evaluations across 12 COP settings show that 1) verbal reflections for evolution lead to smoother fitness landscapes, explicit inference of black-box COP settings, and better search results; 2) heuristics generated by ReEvo in minutes can outperform state-of-the-art human designs and neural solvers; 3) LHHs enable efficient algorithm design automation even when challenged with black-box COPs, demonstrating its potential for complex and novel real-world applications. Our code is available: https://github.com/ai4co/LLM-as-HH.

翻译：NP难组合优化问题（COPs）的普遍存在迫使领域专家反复进行启发式设计的试错过程。随着大语言模型（LLMs）的兴起，设计自动化这一长期目标获得了新的动力。本文提出语言超启发式（LHHs）——一种新兴的超启发式变体，利用LLMs进行启发式生成，具有最小人工干预和开放式启发式空间的特点。为增强LHHs，我们提出反思进化（ReEvo）通用搜索框架，该框架模拟人类专家的反思式设计方法，同时凭借其可扩展的LLM推理、互联网规模的领域知识与强大的进化搜索，远超人类能力上限。在12种COP场景下的评估表明：1）面向进化的语言反思能够带来更平滑的适应度景观、实现对黑箱COP场景的显式推断以及更优的搜索结果；2）ReEvo在数分钟内生成的启发式算法，其性能可超越当前最先进的人工设计与神经求解器；3）即使面对黑箱COPs，LHHs仍能实现高效的算法设计自动化，展现出其在复杂新颖的真实世界应用中的潜力。我们的代码已开源：https://github.com/ai4co/LLM-as-HH