Deep research models perform multi-step research to produce long-form, well-attributed answers. However, most open deep research models are trained on easily verifiable short-form QA tasks via reinforcement learning with verifiable rewards (RLVR), which does not extend to realistic long-form tasks. We address this with Reinforcement Learning with Evolving Rubrics (RLER), in which we construct and maintain rubrics that co-evolve with the policy model during training; this allows the rubrics to incorporate information that the model has newly explored and to provide discriminative, on-policy feedback. Using RLER, we develop Deep Research Tulu (DR Tulu-8B), the first open model that is directly trained for open-ended, long-form deep research. Across four long-form deep research benchmarks in science, healthcare and general domains, DR Tulu substantially outperforms existing open deep research models, and matches or exceeds proprietary deep research systems, while being significantly smaller and cheaper per query. To facilitate future research, we release all data, models, and code, including our new MCP-based agent infrastructure for deep research systems.

翻译：深度研究模型通过多步骤研究来生成长篇、可充分溯源的回答。然而，大多数开放的深度研究模型是通过带有可验证奖励的强化学习（RLVR）在易于验证的短篇问答任务上训练的，这无法扩展到现实的长篇任务。我们通过基于演化评分标准的强化学习（RLER）来解决这一问题，在该方法中，我们构建并维护在训练过程中与策略模型协同演化的评分标准；这使得评分标准能够整合模型新探索的信息，并提供具有区分度的、基于当前策略的反馈。利用RLER，我们开发了深度研究Tulu（DR Tulu-8B），这是首个为开放式长篇深度研究直接训练的开放模型。在科学、医疗保健和通用领域的四个长篇深度研究基准测试中，DR Tulu显著优于现有的开放深度研究模型，并匹配或超越了专有的深度研究系统，同时模型规模显著更小且每次查询成本更低。为促进未来研究，我们发布了所有数据、模型和代码，包括我们用于深度研究系统的新MCP智能体基础设施。