Large language Model (LLM)-assisted algorithm discovery is an iterative, black-box optimization process over programs to approximatively solve a target task, where an LLM proposes candidate programs and an external evaluator provides task feedback. Despite intense recent research on the topic and promising results, how can the LLM internal representation of the space of possible programs be maximally exploited to improve performance is an open question. Here, we introduce Contrastive Concept-Tree Search (CCTS), which extracts a hierarchical concept representation from the generated programs and learns a contrastive concept model that guides parent selection. By reweighting parents using a likelihood-ratio score between high- and low-performing solutions, CCTS biases search toward useful concept combinations and away from misleading ones, providing guidance through an explicit concept hierarchy rather than the algorithm lineage constructed by the LLM. We show that CCTS improves search efficiency over fitness-based baselines and produces interpretable, task-specific concept trees across a benchmark of open Erdős-type combinatorics problems. Our analysis indicates that the gains are driven largely by learning which concepts to avoid. We further validate these findings in a controlled synthetic algorithm-discovery environment, which reproduces qualitatively the search dynamics observed with the LLMs.

翻译：大型语言模型（LLM）辅助的算法发现是一个基于程序的黑盒优化迭代过程，旨在近似求解目标任务，其中LLM提出候选程序，外部评估器提供任务反馈。尽管近期该领域研究活跃且成果显著，但如何最大限度地利用LLM对可能程序空间的内在表示以提升性能，仍是一个悬而未决的问题。本文提出对比式概念树搜索（CCTS），该方法从生成的程序中提取层次化概念表示，并学习一个指导父节点选择的对比式概念模型。通过对高绩效解与低绩效解之间的似然比得分重新加权父节点，CCTS将搜索偏向有益的概念组合并远离误导性组合，通过显式的概念层次结构而非LLM构建的算法谱系提供指导。我们证明，在开放式埃尔德什型组合问题基准测试中，CCTS相比基于适应度的基线方法提升了搜索效率，并生成了可解释的、任务特定的概念树。分析表明，性能提升主要源于学习应避免哪些概念。我们进一步在受控的合成算法发现环境中验证了这些发现，该环境定性地复现了LLM中观察到的搜索动态。