Test-Time Reinforcement Learning (TTRL) enables Large Language Models (LLMs) to enhance reasoning capabilities on unlabeled test streams by deriving pseudo-rewards from majority voting consensus. However, existing TTRL methods rely exclusively on positive pseudo-labeling strategies. Such reliance becomes vulnerable under challenging scenarios where answer distributions are highly dispersed, resulting in weak consensus that inadvertently reinforces incorrect trajectories as supervision signals. In this paper, we propose SCRL (Selective-Complementary Reinforcement Learning), a robust test-time reinforcement learning framework that effectively mitigates label noise amplification. SCRL develops Selective Positive Pseudo-Labeling, which enforces strict consensus criteria to filter unreliable majorities. Complementarily, SCRL introduces Entropy-Gated Negative Pseudo-Labeling, the first negative supervision mechanism in TTRL, to reliably prune incorrect trajectories based on generation uncertainty. Extensive experiments on multiple reasoning benchmarks demonstrate that SCRL achieves substantial improvements over baselines, while maintaining robust generalization and training stability under constrained rollout budgets. Our code is available at https://github.com/Jasper-Yan/SCRL.
翻译:测试时强化学习(TTRL)通过从多数投票共识中推导伪奖励,使大型语言模型(LLM)能够在未标记的测试流上增强推理能力。然而,现有的TTRL方法完全依赖于正向伪标注策略。这种依赖在答案分布高度分散的挑战性场景下变得脆弱,导致弱共识会无意中强化错误轨迹作为监督信号。本文提出SCRL(选择性互补强化学习),一种稳健的测试时强化学习框架,有效缓解标签噪声放大问题。SCRL开发了选择性正向伪标注机制,通过严格的共识标准过滤不可靠的多数结果;作为补充,SCRL引入了熵门控负向伪标注——首个TTRL中的负向监督机制——基于生成不确定性可靠地剪枝错误轨迹。在多个推理基准上的大量实验表明,SCRL相较于基线方法取得了显著提升,同时在受限的回滚预算下保持稳健的泛化能力和训练稳定性。我们的代码已开源:https://github.com/Jasper-Yan/SCRL