LLMs utilizing chain-of-thought reasoning often waste substantial compute by producing long, incorrect responses. Abstention can mitigate this by withholding outputs unlikely to be correct. While most abstention methods decide to withhold outputs before or after generation, dynamic mid-generation abstention considers early termination of unpromising reasoning traces at each token position. Prior work has explored empirical variants of this idea, but principled guidance for the abstention rule remains lacking. We present a formal analysis of dynamic abstention for LLMs, modeling abstention as an explicit action within a regularized reinforcement learning framework. An abstention reward parameter controls the trade-off between compute and information. We show that abstaining when the value function falls below this reward strictly outperforms natural baselines under general conditions. We further derive a principled and efficient method to approximate the value function. Empirical results on mathematical reasoning and toxicity avoidance tasks support our theory and demonstrate improved selective accuracy over existing methods.

翻译：利用思维链推理的大语言模型常因生成冗长且错误的响应而浪费大量计算资源。弃权策略通过拒绝输出可能错误的答案可缓解此问题。现有弃权方法多在生成前或生成后决定是否输出，而动态中途弃权则考虑在每个词元位置提前终止无希望的推理轨迹。虽然已有研究探索过此思想的经验变体，但弃权规则的理论指导仍显匮乏。本文对LLM动态弃权进行形式化分析，将弃权建模为带正则化的强化学习框架中的显式动作。弃权奖励参数控制计算量与信息量之间的权衡。研究表明，在通用条件下，当价值函数低于该奖励时弃权，其性能严格优于基线方法。我们进一步推导出近似价值函数的理论化高效方法。在数学推理与毒性规避任务上的实证结果支持我们的理论，并证明相较于现有方法具有更高的选择性准确率。