Agentic search enables language models to solve knowledge-intensive tasks by adaptively acquiring external evidence over multiple steps. Reinforcement learning with verifiable rewards (RLVR) has emerged as a widely adopted training paradigm for search agents, yet outcome-only rewards are sparse and provide limited credit assignment for intermediate search actions. Existing process-reward methods therefore seek to densify supervision through proxy signals, external evaluators, or likelihood-based information gain. However, proxy rewards can deviate from the final outcome objective, while fixed evaluators can become stale as the search policy evolves, leading to unreliable process supervision. To address these challenges, we propose OASES, an Outcome-Aligned Search-Evaluation Supervision framework for agentic search. OASES derives outcome-aligned process rewards by evaluating how well each intermediate search state supports answering the original question. It further co-trains the search policy and the state evaluator on policy, allowing the evaluator to adapt to evolving search behavior and provide more reliable process rewards. Experiments on five multi-hop QA benchmarks show that OASES consistently outperforms strong RL baselines, with further analyses confirming the benefits of outcome-aligned process rewards and search-evaluation co-training.

翻译：智能搜索使语言模型能够通过多步自适应获取外部证据来解决知识密集型任务。基于可验证奖励的强化学习（RLVR）已成为搜索代理广泛采用的训练范式，但仅依赖最终结果的奖励信号稀疏且对中间搜索动作的信用分配有限。现有过程奖励方法试图通过代理信号、外部评估器或基于似然的信息增益来稠密化监督信号。然而，代理奖励可能偏离最终结果目标，而固定评估器会随着搜索策略的演化变得陈旧，导致过程监督不可靠。为应对这些挑战，我们提出OASES——一种面向智能搜索的结果对齐搜索-评估监督框架。OASES通过评估每个中间搜索状态对回答原始问题的支撑程度，推导出与最终结果对齐的过程奖励。该框架进一步对搜索策略与状态评估器进行联合训练，使评估器能够适应不断演化的搜索行为，提供更可靠的过程奖励。在五个多跳问答基准上的实验表明，OASES持续优于强强化学习基线，进一步的分析证实了结果对齐过程奖励与搜索-评估联合训练的优越性。