Heuristics are widely used for dealing with complex search and optimization problems. However, manual design of heuristics can be often very labour extensive and requires rich working experience and knowledge. This paper proposes Evolution of Heuristic (EoH), a novel evolutionary paradigm that leverages both Large Language Models (LLMs) and Evolutionary Computation (EC) methods for Automatic Heuristic Design (AHD). EoH represents the ideas of heuristics in natural language, termed thoughts. They are then translated into executable codes by LLMs. The evolution of both thoughts and codes in an evolutionary search framework makes it very effective and efficient for generating high-performance heuristics. Experiments on three widely studied combinatorial optimization benchmark problems demonstrate that EoH outperforms commonly used handcrafted heuristics and other recent AHD methods including FunSearch. Particularly, the heuristic produced by EoH with a low computational budget (in terms of the number of queries to LLMs) significantly outperforms widely-used human hand-crafted baseline algorithms for the online bin packing problem.

翻译：启发式算法被广泛用于处理复杂的搜索与优化问题。然而，手动设计启发式算法通常非常耗时，且需要丰富的工作经验与专业知识。本文提出启发式算法演化（Evolution of Heuristic, EoH），这是一种新颖的演化范式，它结合大型语言模型（LLMs）与演化计算（EC）方法来实现自动启发式设计（AHD）。EoH 使用自然语言表示启发式算法的设计思想，称为“思维”。这些思维随后由 LLMs 翻译为可执行代码。在演化搜索框架中，思维与代码的共同演化使得该方法能够高效地生成高性能启发式算法。在三个被广泛研究的组合优化基准问题上的实验表明，EoH 的性能优于常用的手工设计启发式算法以及其他近期 AHD 方法（包括 FunSearch）。特别地，在较低计算预算（以向 LLMs 的查询次数衡量）下，EoH 生成的启发式算法在在线装箱问题上显著优于广泛使用的人工设计基线算法。