While Reinforcement Learning with Verifiable Rewards (RLVR) has advanced the reasoning capabilities of Large Vision-Language Models (LVLMs), most existing methods in multimodal reasoning neglect the critical role of visual perception within the RLVR optimization process. In this paper, we undertake a pioneering exploration of multimodal RLVR through the novel perspective of token perception, which measures the visual dependency of each generated token. With a granular analysis of Chain-of-Thought (CoT) processes, we uncover two key insights: first, token perception in a rollout trajectory is sparsely distributed, where only a small fraction of tokens have high visual dependency for visually-grounded reasoning; second, different trajectories exhibit significant divergence in their overall visual dependency. Based on these observations, we propose Visually-Perceptive Policy Optimization (VPPO), a novel policy gradient algorithm that explicitly leverages token perception to refine the learning signal. Specifically, VPPO achieves this through a dual mechanism: it reweights a trajectory's advantage by its overall visual dependency, and focuses policy updates exclusively on perceptually pivotal tokens. On a comprehensive suite of eight perception and reasoning benchmarks, VPPO demonstrates substantial gains over leading open-source RL-tuned models, with its effectiveness consistently validated across 7B and 32B model scales. Our findings not only establish a new token-level perceptual perspective for analyzing multimodal RLVR but also present a novel and effective optimization strategy to significantly enhance the multimodal reasoning capabilities of LVLMs.

翻译：尽管可验证奖励强化学习（RLVR）提升了大型视觉语言模型（LVLMs）的推理能力，但现有多模态推理方法大多忽视了视觉感知在RLVR优化过程中的关键作用。本文从令牌感知这一新颖视角，首次对多模态RLVR进行了开创性探索——该视角通过衡量每个生成令牌的视觉依赖性来量化感知程度。通过对思维链（CoT）过程的细粒度分析，我们揭示了两项关键发现：首先，在展开轨迹中令牌感知呈稀疏分布，仅少数令牌对基于视觉的推理具有高依赖性；其次，不同轨迹在整体视觉依赖性上表现出显著差异。基于这些观察，我们提出了视觉感知策略优化（VPPO），这是一种新颖的策略梯度算法，其显式利用令牌感知来优化学习信号。具体而言，VPPO通过双重机制实现：依据轨迹的整体视觉依赖性重新加权其优势值，并仅对感知关键令牌进行策略更新。在涵盖八个感知与推理基准的综合测试中，VPPO相较于领先的开源RL调优模型取得了显著提升，其有效性在7B和32B模型规模上均得到一致验证。我们的研究不仅为分析多模态RLVR建立了全新的令牌级感知视角，更提出了一种能显著增强LVLMs多模态推理能力的新型高效优化策略。