As large language model agents advance beyond software engineering (SWE) tasks toward machine learning engineering (MLE), verifying agent behavior becomes orders of magnitude more expensive: while SWE tasks can be verified via fast-executing unit tests, MLE verification requires running full ML pipelines -- data preprocessing, model training, and metric evaluation -- on large datasets at each rollout step, rendering trajectory-wise on-policy reinforcement learning (RL) prohibitively slow. Existing approaches retreat to supervised fine-tuning (SFT) or offline proxy rewards, sacrificing the exploration and generalization benefits of on-policy RL. We observe that sandbox data size is the primary source of this bottleneck. Based on this insight, we introduce SandMLE, a multi-agent framework that generates diverse, verifiable synthetic MLE environments from a small number of seed tasks, preserving the structural and technical complexity of real-world problems while constraining datasets to micro-scale (each task is paired with only 50-200 training samples). Through extensive experiments, we show that SandMLE reduces execution time by over 13 times, enabling large-scale, on-policy trajectory-wise RL for the first time in the MLE domain. On MLE-bench-lite, SandMLE yields significant gains over SFT baselines across Qwen3-8B, 14B, and 30B-A3B, with relative medal rate improvements ranging from 20.3% to 66.9%. Furthermore, the trained policy generalizes across unseen agentic scaffolds, achieving up to 32.4% better HumanRank score on MLE-Dojo.

翻译：随着大语言模型智能体从软件工程（SWE）任务向机器学习工程（MLE）领域迈进，验证智能体行为的成本呈数量级增长：SWE任务可通过快速执行的单元测试验证，而MLE验证则需要每次策略更新时，在大型数据集上运行完整的机器学习流水线——包括数据预处理、模型训练和指标评估，这使得基于轨迹的在策略强化学习（RL）变得极其缓慢。现有方法退而求其次采用监督微调（SFT）或离线代理奖励，牺牲了在策略强化学习的探索与泛化优势。我们发现沙箱数据规模是这一瓶颈的主要来源。基于这一洞察，我们提出SandMLE，这是一个多智能体框架，能够从少量种子任务生成多样且可验证的合成MLE环境，在保持现实问题结构与技术复杂性的同时，将数据集规模限制在微观级别（每个任务仅配备50-200个训练样本）。通过大量实验表明，SandMLE将执行时间降低了13倍以上，首次在MLE领域实现了大规模、基于轨迹的在策略强化学习。在MLE-bench-lite基准上，SandMLE在Qwen3-8B、14B和30B-A3B模型上均显著优于SFT基线方法，奖牌率相对提升幅度达20.3%至66.9%。此外，训练后的策略可泛化至未见过的智能体框架，在MLE-Dojo上HumanRank分数最高提升32.4%。