Large Language Models (LLMs) have advanced Automated Heuristic Design (AHD) in combinatorial optimization (CO) in the past few years. However, existing discovery pipelines often require extensive manual trial-and-error or reliance on domain expertise to adapt to new or complex problems. This stems from tightly coupled internal mechanisms that limit systematic improvement of the LLM-driven design process. To address this challenge, we propose a structured framework for LLM-driven AHD that explicitly decomposes the heuristic discovery process into modular stages: a forward pass for evaluation, a backward pass for analytical feedback, and an update step for program refinement. This separation provides a clear abstraction for iterative refinement and enables principled improvements of individual components. We validate our framework across four diverse real-world CO domains, where it consistently outperforms baselines, achieving up to $0.17$ improvement in QYI on unseen test sets. Finally, we show that several popular AHD methods are restricted instantiations of our framework. By integrating them in our structured pipeline, we can upgrade the components modularly and significantly improve their performance.

翻译：过去几年中，大型语言模型（LLMs）在组合优化（CO）领域的自动启发式设计（AHD）方面取得了显著进展。然而，现有的发现流程通常需要大量手动试错或依赖领域专业知识来适应新的或复杂的问题。这源于其紧密耦合的内部机制，限制了LLM驱动设计过程的系统性改进。为应对这一挑战，我们提出了一个用于LLM驱动AHD的结构化框架，该框架将启发式发现过程显式分解为模块化阶段：用于评估的前向传播、用于分析反馈的反向传播，以及用于程序优化的更新步骤。这种分离为迭代优化提供了清晰的抽象，并支持对各个组件进行有原则的改进。我们在四个不同的现实世界CO领域中验证了该框架，其表现始终优于基线方法，在未见测试集上实现了高达$0.17$的QYI提升。最后，我们证明若干流行的AHD方法均可视为本框架的受限实例化。通过将其纳入我们的结构化流程，我们可以模块化地升级组件并显著提升其性能。