Existing benchmarks for LLM coding agents primarily evaluate final outcomes. While useful for measuring overall capability, these metrics provide limited visibility and often miss defects that arise during execution. We present ProcCtrlBench, a benchmark for execution-process evaluation in LLM coding agents. ProcCtrlBench organizes recurrent execution defects into a reusable ontology covering 11 defect types in 4 categories, and evaluates agent trajectories through standardized process evidence rather than final outcomes alone. To support comparison across heterogeneous agents, ProcCtrlBench standardizes raw logs into a unified trajectory representation and reports calibrated scorecards over process-level findings. In addition, ProcCtrlBench uses control preservation as a way to quantify execution-process quality, capturing whether execution remains interpretable, interruptible, correctable, reversible, and able to hand back authority when needed. We evaluate ProcCtrlBench on 200 cases sampled from three benchmarks: AndroidBench, TerminalBench, and SWE-bench-Verified. Results show that ProcCtrlBench can be instantiated with useful reliability, provides more stable semantics than direct thresholding, and reveals meaningful differences in execution quality that are often overlooked by conventional outcome-based evaluation.

翻译：现有的大语言模型编码代理基准测试主要评估最终结果。虽然这对于衡量整体能力有用，但这些指标提供的可见性有限，且常常遗漏执行过程中出现的缺陷。我们提出ProcCtrlBench，一个用于大语言模型编码代理执行过程评估的基准测试。ProcCtrlBench将周期性执行缺陷组织成一个可复用的本体，涵盖4个类别中的11种缺陷类型，并通过标准化的过程证据（而非仅基于最终结果）来评估代理的执行轨迹。为了支持异构代理间的比较，ProcCtrlBench将原始日志标准化为统一的轨迹表示形式，并针对过程级发现报告校准后的评分卡。此外，ProcCtrlBench采用可控性保持作为量化执行过程质量的一种方式，捕获执行是否保持可解释、可中断、可修正、可逆转，以及在需要时能否归还控制权。我们在从三个基准测试（AndroidBench、TerminalBench和SWE-bench-Verified）中抽样的200个案例上评估了ProcCtrlBench。结果表明，ProcCtrlBench能以有用的可靠性进行实例化，提供比直接阈值法更稳定的语义，并揭示了传统基于结果的评估常忽略的执行质量方面的显著差异。