Large Language Models (LLMs) have emerged as powerful operators for evolutionary search, yet the design of efficient search scaffolds remains ad hoc. While promising, current LLM-in-the-loop systems lack a systematic approach to managing the evolutionary process. We identify three distinct failure modes: Context Pollution, where experiment history biases future candidate generation; Mode Collapse, where agents stagnate in local minima due to poor exploration-exploitation balance; and Weak Collaboration, where rigid crossover strategies fail to leverage parallel search trajectories effectively. We introduce Progress-Aware Consistent Evolution (PACEvolve), a framework designed to robustly govern the agent's context and search dynamics, to address these challenges. PACEvolve combines hierarchical context management (HCM) with pruning to address context pollution; momentum-based backtracking (MBB) to escape local minima; and a self-adaptive sampling policy that unifies backtracking and crossover for dynamic search coordination (CE), allowing agents to balance internal refinement with cross-trajectory collaboration. We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement, achieving state-of-the-art results on LLM-SR and KernelBench, while discovering solutions surpassing the record on Modded NanoGPT.

翻译：大语言模型已成为进化搜索的强大算子，但高效搜索框架的设计仍处于临时性阶段。尽管当前基于大语言模型的循环系统展现出潜力，但其缺乏管理演化过程的系统性方法。我们识别出三种典型的失效模式：上下文污染，即实验历史对未来候选生成产生偏差；模式坍塌，即智能体因探索-利用平衡失调而陷入局部最优；以及弱协作，即僵化的交叉策略无法有效利用并行搜索轨迹。为应对这些挑战，我们提出进度感知一致性演化框架，该框架通过稳健控制智能体上下文与搜索动态来解决问题。PACEvolve融合了分层上下文管理与剪枝机制以应对上下文污染；采用动量回溯法以逃离局部最优；并引入自适应采样策略，将回溯与交叉统一为动态搜索协调机制，使智能体能够平衡内部优化与跨轨迹协作。实验证明，PACEvolve为持续长视野自我改进提供了系统性路径，在LLM-SR和KernelBench基准上取得最先进成果，并在Modded NanoGPT任务上发现了超越历史记录的解决方案。