Verification is critical for improving agents: it provides the reward signal for Reinforcement Learning and enables inference-time gains through Test-Time Scaling (TTS). Despite its importance, verification in software engineering (SWE) agent settings often relies on code execution, which can be difficult to scale due to environment setup overhead. Scalable alternatives such as patch classifiers and heuristic methods exist, but they are less grounded in codebase context and harder to interpret. To this end, we explore Agentic Rubrics: an expert agent interacts with the repository to create a context-grounded rubric checklist, and candidate patches are then scored against it without requiring test execution. On SWE-Bench Verified under parallel TTS evaluation, Agentic Rubrics achieve a score of 54.2% on Qwen3-Coder-30B-A3B and 40.6% on Qwen3-32B, with at least a +3.5 percentage-point gain over the strongest baseline in our comparison set. We further analyze rubric behavior, showing that rubric scores are consistent with ground-truth tests while also flagging issues that tests do not capture. Our ablations show that agentic context gathering is essential for producing codebase-specific, unambiguous criteria. Together, these results suggest that Agentic Rubrics provide an efficient, scalable, and granular verification signal for SWE agents.

翻译：验证对于改进代理至关重要：它为强化学习提供奖励信号，并通过测试时缩放（TTS）实现推理时的性能提升。尽管验证至关重要，但在软件工程（SWE）代理场景中，验证通常依赖于代码执行，而由于环境设置的开销，这种方法难以扩展。存在可扩展的替代方案，如补丁分类器和启发式方法，但它们较少基于代码库上下文且更难以解释。为此，我们探索了智能量规方法：一个专家代理与代码仓库交互，创建一个基于上下文的量规检查清单，然后根据该清单对候选补丁进行评分，而无需执行测试。在并行TTS评估下的SWE-Bench Verified基准测试中，智能量规在Qwen3-Coder-30B-A3B上取得了54.2%的分数，在Qwen3-32B上取得了40.6%的分数，比我们比较集中最强的基线至少高出3.5个百分点。我们进一步分析了量规的行为，表明量规评分与真实测试结果一致，同时还能标记出测试未能捕捉到的问题。我们的消融实验表明，智能上下文收集对于生成特定于代码库的、明确的评判标准至关重要。综上所述，这些结果表明智能量规为软件工程代理提供了一种高效、可扩展且细粒度的验证信号。