Large Language Models (LLMs) have emerged as powerful operators for evolutionary search, yet the design of efficient search scaffolds remains ad hoc. While promising, current LLM-in-the-loop systems lack a systematic approach to managing the evolutionary process. We identify three distinct failure modes: Context Pollution, where experiment history biases future candidate generation; Mode Collapse, where agents stagnate in local minima due to poor exploration-exploitation balance; and Weak Collaboration, where rigid crossover strategies fail to leverage parallel search trajectories effectively. We introduce Progress-Aware Consistent Evolution (PACEvolve), a framework designed to robustly govern the agent's context and search dynamics, to address these challenges. PACEvolve combines hierarchical context management (HCM) with pruning to address context pollution; momentum-based backtracking (MBB) to escape local minima; and a self-adaptive sampling policy that unifies backtracking and crossover for dynamic search coordination (CE), allowing agents to balance internal refinement with cross-trajectory collaboration. We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement, achieving state-of-the-art results on LLM-SR and KernelBench, while discovering solutions surpassing the record on Modded NanoGPT.

翻译：大型语言模型（LLM）已成为进化搜索的强大操作者，但高效搜索框架的设计仍处于临时性阶段。尽管前景广阔，当前基于LLM的循环系统缺乏管理演化过程的系统性方法。我们识别出三种典型的失效模式：上下文污染，即实验历史对未来候选生成产生偏差；模式坍塌，即智能体因探索-利用平衡不佳而陷入局部最优；以及弱协作，即僵化的交叉策略无法有效利用并行搜索轨迹。为应对这些挑战，我们提出了进展感知一致性演化（PACEvolve）框架，旨在稳健地管理智能体的上下文与搜索动态。PACEvolve通过以下机制整合解决问题：采用分层上下文管理（HCM）与剪枝策略应对上下文污染；基于动量的回溯（MBB）机制以逃离局部最优；以及自适应采样策略，该策略将回溯与交叉统一为动态搜索协调（CE），使智能体能够平衡内部优化与跨轨迹协作。实验表明，PACEvolve为持续、长时程的自我改进提供了系统性路径，在LLM-SR和KernelBench基准上取得了最先进的结果，并在Modded NanoGPT任务上发现了超越现有记录的解决方案。