Reinforcement Learning with Verifiable Rewards (RLVR) elicits long chain-of-thought reasoning in large language models (LLMs), but outcome-based rewards lead to coarse-grained advantage estimation. While existing approaches improve RLVR via token-level entropy or sequence-level length control, they lack a semantically grounded, step-level measure of reasoning progress. As a result, LLMs fail to distinguish necessary deduction from redundant verification: they may continue checking after reaching a correct solution and, in extreme cases, overturn a correct trajectory into an incorrect final answer. To remedy the lack of process supervision, we introduce a training-free probing mechanism that extracts intermediate confidence and correctness and combines them into a Step Potential signal that explicitly estimates the reasoning state at each step. Building on this signal, we propose Step Potential Advantage Estimation (SPAE), a fine-grained credit assignment method that amplifies potential gains, penalizes potential drops, and applies penalty after potential saturates to encourage timely termination. Experiments across multiple benchmarks show SPAE consistently improves accuracy while substantially reducing response length, outperforming strong RL baselines and recent efficient reasoning and token-level advantage estimation methods. The code is available at https://github.com/cii030/SPAE-RL.

翻译：基于可验证奖励的强化学习（RLVR）能够激发大型语言模型（LLM）进行长链式思维推理，但基于结果的奖励会导致粗粒度的优势估计。现有方法通过词元级熵或序列级长度控制来改进RLVR，但缺乏一种语义上基于推理进度的步骤级度量。因此，LLM难以区分必要的推导与冗余的验证：它们可能在得到正确解后继续检查，甚至在极端情况下将正确的推理轨迹推翻为错误的最终答案。为弥补过程监督的缺失，我们引入一种免训练的探测机制，该机制提取中间置信度与正确性，并将其组合成一个步骤潜能信号，用以显式估计每一步的推理状态。基于此信号，我们提出了步骤潜能优势估计（SPAE），这是一种细粒度的信用分配方法，能够放大潜在增益、惩罚潜在下降，并在潜能饱和后施加惩罚以鼓励及时终止。在多个基准测试上的实验表明，SPAE在显著减少响应长度的同时，持续提升准确率，其性能优于强RL基线以及近期的高效推理与词元级优势估计方法。代码发布于 https://github.com/cii030/SPAE-RL。