Heuristics are commonly used to tackle diverse search and optimization problems. Design heuristics usually require tedious manual crafting with domain knowledge. Recent works have incorporated large language models (LLMs) into automatic heuristic search leveraging their powerful language and coding capacity. However, existing research focuses on the optimal performance on the target problem as the sole objective, neglecting other criteria such as efficiency and scalability, which are vital in practice. To tackle this challenge, we propose to model heuristic search as a multi-objective optimization problem and consider introducing other practical criteria beyond optimal performance. Due to the complexity of the search space, conventional multi-objective optimization methods struggle to effectively handle multi-objective heuristic search. We propose the first LLM-based multi-objective heuristic search framework, Multi-objective Evolution of Heuristic (MEoH), which integrates LLMs in a zero-shot manner to generate a non-dominated set of heuristics to meet multiple design criteria. We design a new dominance-dissimilarity mechanism for effective population management and selection, which incorporates both code dissimilarity in the search space and dominance in the objective space. MEoH is demonstrated in two well-known combinatorial optimization problems: the online Bin Packing Problem (BPP) and the Traveling Salesman Problem (TSP). Results indicate that a variety of elite heuristics are automatically generated in a single run, offering more trade-off options than existing methods. It successfully achieves competitive or superior performance while improving efficiency up to 10 times. Moreover, we also observe that the multi-objective search introduces novel insights into heuristic design and leads to the discovery of diverse heuristics.

翻译：启发式方法常用于解决各类搜索与优化问题。设计启发式通常需要结合领域知识进行繁琐的手工构建。近期研究利用大语言模型（LLMs）强大的语言与代码生成能力，将其引入自动启发式搜索过程。然而，现有研究仅以目标问题的最优性能为单一优化目标，忽视了效率与可扩展性等实际应用中至关重要的其他准则。为应对这一挑战，本文提出将启发式搜索建模为多目标优化问题，并考虑引入除最优性能外的其他实践准则。由于搜索空间复杂度高，传统多目标优化方法难以有效处理多目标启发式搜索。我们提出了首个基于LLM的多目标启发式搜索框架——多目标启发式进化（MEoH），该框架以零样本方式集成LLM，生成满足多重设计准则的非支配启发式集合。我们设计了一种新的支配-相异性机制用于有效的种群管理与选择，该机制同时结合了搜索空间中的代码相异性与目标空间中的支配关系。MEoH在两个经典组合优化问题中得到验证：在线装箱问题（BPP）与旅行商问题（TSP）。实验结果表明，单次运行即可自动生成多种优质启发式，相比现有方法提供更多权衡选择。该方法在实现竞争性或更优性能的同时，将效率提升最高达10倍。此外，我们还观察到多目标搜索为启发式设计带来了新的见解，并促进了多样化启发式的发现。