Autonomous LLM training is often framed as recipe search, which leaves the training harness largely static. This limitation sharpens in agentic RL, where shifting bottlenecks and scalar rewards mask diverse failure modes. We introduce EvoTrainer, an autonomous training framework that co-evolves LLM policies and training-side harnesses through empirical feedback: it diagnoses rollout-level evidence, revises diagnostics, backtests interventions, and accumulates reusable skills. Evaluated on mathematical reasoning, competitive-programming code generation, and repository-level software engineering, EvoTrainer matches or exceeds the human-engineered RL references under the same data, codebase, and evaluation protocol, with the largest gain on long-horizon agentic SWE. Trajectory analyses show that retained strategies diverge across domains, evolving diagnostics prevent invalid high-scoring branches from being promoted, and reusable skills shape later search. Autonomous LLM RL should move beyond recipe search toward joint evolution of policies and the training harnesses that interpret them.

翻译：自主大语言模型训练常被定义为配方搜索问题，这使得训练框架基本保持静态。这一局限在智能体强化学习场景中更为突出——动态变化的性能瓶颈与标量奖励信号会掩盖多样的失败模式。本文提出EvoTrainer自主训练框架，通过经验反馈协同进化策略与训练侧框架：诊断轨迹级证据、修正诊断结论、回测干预措施并积累可复用技能。在数学推理、竞赛编程代码生成及仓库级软件工程任务上的评估表明，相同数据、代码库与评估协议条件下，EvoTrainer的性能持平或超越人工设计的强化学习基线，其中长周期智能体软件工程任务提升最为显著。轨迹分析显示：不同领域保留策略呈现分化，进化式诊断可阻止获得高分的无效分支被提升，可复用技能将影响后续搜索。自主大语言模型强化学习应从配方搜索转向策略与解释策略的训练框架的联合进化。