Web agents hold great potential for automating complex computer tasks, yet their interactions involve long-horizon, sequential decision-making with irreversible actions. In such settings, outcome-based supervision is sparse and delayed, often rewarding incorrect trajectories and failing to support inference-time scaling. This motivates the use of Process Reward Models (WebPRMs) for web navigation, but existing approaches remain limited: scalar WebPRMs collapse progress into coarse, weakly grounded signals, while checklist-based WebPRMs rely on brittle template matching that fails under layout or semantic changes and often mislabels superficially correct actions as successful, providing little insight or interpretability. To address these challenges, we introduce WebArbiter, a reasoning-first, principle-inducing WebPRM that formulates reward modeling as text generation, producing structured justifications that conclude with a preference verdict and identify the action most conducive to task completion under the current context. Training follows a two-stage pipeline: reasoning distillation equips the model with coherent principle-guided reasoning, and reinforcement learning corrects teacher biases by directly aligning verdicts with correctness, enabling stronger generalization. To support systematic evaluation, we release WebPRMBench, a comprehensive benchmark spanning four diverse web environments with rich tasks and high-quality preference annotations. On WebPRMBench, WebArbiter-7B outperforms the strongest baseline, GPT-5, by 9.1 points. In reward-guided trajectory search on WebArena-Lite, it surpasses the best prior WebPRM by up to 6.4 points, underscoring its robustness and practical value in complex web tasks.

翻译：网页代理在自动化复杂计算机任务方面潜力巨大，但其交互涉及长程、序列化决策过程，且包含不可逆操作。在此类场景中，基于结果的监督信号稀疏且延迟，常导致错误轨迹获得奖励，并难以支持推理时扩展。这促使人们采用过程奖励模型（WebPRM）来辅助网页导航，但现有方法仍存在局限：标量型WebPRM将进度压缩为粗粒度的弱监督信号，而基于检查表的WebPRM依赖脆弱的模板匹配，在布局或语义变化时容易失效，且常将表面正确的动作误判为成功，缺乏洞察力与可解释性。为应对这些挑战，我们提出WebArbiter——一种以推理优先、准则归纳为核的WebPRM，其将奖励建模转化为文本生成任务，输出结构化论证后附偏好性判定结论，并识别出当前上下文中最利于任务完成的动作。训练采用两阶段流水线：推理蒸馏赋予模型连贯的准则引导推理能力，强化学习则通过直接对齐判定结论与正确性来修正教师偏差，从而增强泛化性能。为支撑系统评估，我们发布WebPRMBench基准测试集，覆盖四个多样化网页环境，包含丰富任务与高质量偏好标注。在WebPRMBench上，WebArbiter-7B以9.1分的优势超越最强基线GPT-5；在WebArena-Lite的奖励引导轨迹搜索中，其性能较此前最优WebPRM提升最高达6.4分，充分彰显了复杂网页任务中的鲁棒性与实用价值。