Large language models (LLMs) are increasingly adapted to downstream tasks via reinforcement learning (RL) methods like Group Relative Policy Optimization (GRPO), which often require thousands of rollouts to learn new tasks. We argue that the interpretable nature of language often provides a much richer learning medium for LLMs, compared to policy gradients derived from sparse, scalar rewards. To test this, we introduce GEPA (Genetic-Pareto), a prompt optimizer that thoroughly incorporates natural language reflection to learn high-level rules from trial and error. Given any AI system containing one or more LLM prompts, GEPA samples trajectories (e.g., reasoning, tool calls, and tool outputs) and reflects on them in natural language to diagnose problems, propose and test prompt updates, and combine complementary lessons from the Pareto frontier of its own attempts. As a result of GEPA's design, it can often turn even just a few rollouts into a large quality gain. Across six tasks, GEPA outperforms GRPO by 6% on average and by up to 20%, while using up to 35x fewer rollouts. GEPA also outperforms the leading prompt optimizer, MIPROv2, by over 10% (e.g., +12% accuracy on AIME-2025), and demonstrates promising results as an inference-time search strategy for code optimization. We release our code at https://github.com/gepa-ai/gepa .

翻译：大型语言模型（LLM）越来越多地通过强化学习（RL）方法（如群体相对策略优化（GRPO））适应下游任务，这些方法通常需要数千次轨迹采样才能学会新任务。我们认为，与从稀疏标量奖励推导出的策略梯度相比，语言的可解释性通常为LLM提供了更丰富的学习媒介。为验证这一点，我们提出了GEPA（遗传-帕累托），这是一种提示优化器，它充分结合自然语言反思，从试错中学习高级规则。给定任何包含一个或多个LLM提示的AI系统，GEPA对轨迹（例如推理、工具调用和工具输出）进行采样，并以自然语言对其进行反思，以诊断问题、提出并测试提示更新，并从其自身尝试的帕累托前沿中整合互补的经验。得益于GEPA的设计，它通常能将仅有的几次轨迹采样转化为显著的质量提升。在六项任务中，GEPA平均性能超越GRPO 6%，最高可达20%，同时使用的轨迹采样次数减少高达35倍。GEPA还超越了领先的提示优化器MIPROv2超过10%（例如，在AIME-2025上准确率提升+12%），并作为代码优化的推理时搜索策略展现出有前景的结果。我们在https://github.com/gepa-ai/gepa 发布了代码。