A central belief in scaling reinforcement learning with verifiable rewards for instruction following (IF) tasks is that, a diverse mixture of verifiable hard and unverifiable soft constraints is essential for generalizing to unseen instructions. In this work, we challenge this prevailing consensus through a systematic empirical investigation. Counter-intuitively, we find that models trained on hard-only constraints consistently outperform those trained on mixed datasets. Extensive experiments reveal that reward precision, rather than constraint diversity, is the primary driver of effective alignment. The LLM judge suffers from a low recall rate in detecting false response, which leads to severe reward hacking, thereby undermining the benefits of diversity. Furthermore, analysis of the attention mechanism reveals that high-precision rewards develop a transferable meta-skill for IF. Motivated by these insights, we propose a simple yet effective data-centric refinement strategy that prioritizes reward precision. Evaluated on five benchmarks, our approach outperforms competitive baselines by 13.4\% in performance while achieving a 58\% reduction in training time, maintaining strong generalization beyond instruction following. Our findings advocate for a paradigm shift: moving away from the indiscriminate pursuit of data diversity toward high-precision rewards.

翻译：在指令跟随任务中，一个关于利用可验证奖励进行强化学习规模化的核心观点是：可验证的硬约束与不可验证的软约束的多样化混合，对于泛化到未见指令至关重要。在本工作中，我们通过系统的实证研究挑战了这一主流共识。与直觉相反，我们发现，仅使用硬约束训练的模型始终优于在混合数据集上训练的模型。大量实验表明，奖励精度，而非约束多样性，是驱动有效对齐的主要因素。LLM评判器在检测错误响应时召回率较低，这导致了严重的奖励破解，从而削弱了多样性的益处。此外，对注意力机制的分析表明，高精度奖励培养了一种可迁移的指令跟随元技能。基于这些洞见，我们提出了一种简单而有效的数据中心化精炼策略，该策略优先考虑奖励精度。在五个基准测试上的评估表明，我们的方法在性能上优于竞争基线13.4%，同时实现了58%的训练时间缩减，并在指令跟随之外保持了强大的泛化能力。我们的研究结果倡导一种范式转变：从对数据多样性的不加区别追求，转向高精度奖励。