Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) for Large Language Model (LLM) reasoning have been hindered by a persistent challenge: exploration collapse. The semantic homogeneity of random rollouts often traps models in narrow, over-optimized behaviors. While existing methods leverage policy entropy to encourage exploration, they face inherent limitations. Global entropy regularization is susceptible to reward hacking, which can induce meaningless verbosity, whereas local token-selective updates struggle with the strong inductive bias of pre-trained models. To address this, we propose Latent Policy Optimization via Iterative Information Bottleneck (IIB-LPO), a novel approach that shifts exploration from statistical perturbation of token distributions to topological branching of reasoning trajectories. IIB-LPO triggers latent branching at high-entropy states to diversify reasoning paths and employs the Information Bottleneck principle both as a trajectory filter and a self-reward mechanism, ensuring concise and informative exploration. Empirical results across four mathematical reasoning benchmarks demonstrate that IIB-LPO achieves state-of-the-art performance, surpassing prior methods by margins of up to 5.3% in accuracy and 7.4% in diversity metrics.

翻译：具有可验证奖励的强化学习（RLVR）在大语言模型（LLM）推理领域的最新进展一直受到一个持续存在的挑战所阻碍：探索崩溃。随机模拟的语义同质性常常使模型陷入狭窄、过度优化的行为模式。虽然现有方法利用策略熵来鼓励探索，但它们面临着固有的局限性。全局熵正则化容易受到奖励攻击的影响，可能导致无意义的冗长输出，而局部基于令牌的选择性更新则难以克服预训练模型的强归纳偏置。为解决这一问题，我们提出了基于迭代信息瓶颈的潜在策略优化（IIB-LPO），这是一种新颖的方法，它将探索从令牌分布的统计扰动转向推理轨迹的拓扑分支。IIB-LPO在高熵状态触发潜在分支以多样化推理路径，并运用信息瓶颈原则同时作为轨迹过滤器和自奖励机制，确保探索过程简洁且信息丰富。在四个数学推理基准测试上的实证结果表明，IIB-LPO实现了最先进的性能，在准确率上超越先前方法高达5.3%，在多样性指标上超越高达7.4%。