Reinforcement learning with verifiable rewards (RLVR) has successfully elicited the reasoning capabilities of large language models, motivating its extension to multimodal scenarios. Existing methods primarily focus on improving the visual coverage of reasoning traces and mitigating visual hallucinations, but underestimate the semantic inconsistency between the reasoning process and the final answer. In this paper, we delve into thinking-answer inconsistency in RLVR for large vision-language models (LVLMs), showing thorough analyses of rollouts collected throughout Group Relative Policy Optimization (GRPO) training process and post-RLVR evaluation outputs that this issue persists during training and remains present during inference. Motivated by the analysis, we propose Consistency-Oriented Reasoning Alignment (CORA), which introduces thinking-answer semantic consistency into RLVR through a lightweight plug-and-play consistency reward model, and further incorporates Hybrid Reward Advantage Splitting (HRAS) to stably coordinate task and consistency optimization. Extensive experiments across representative multimodal reasoning benchmarks and mainstream LVLMs show that CORA improves task performance while effectively mitigating thinking-answer inconsistency, leading to more faithful reasoning traces.

翻译：基于可验证奖励的强化学习（RLVR）已成功激发大语言模型的推理能力，这推动了其在多模态场景中的扩展。现有方法主要聚焦于改进推理轨迹的视觉覆盖并减轻视觉幻觉，但低估了推理过程与最终答案之间的语义不一致性。本文深入研究了大规模视觉-语言模型（LVLMs）中RLVR的思维-答案不一致性，通过对群组相对策略优化（GRPO）训练过程中收集的展开结果以及后RLVR评估输出的全面分析，表明该问题在训练期间持续存在并在推理阶段依然出现。受分析启发，我们提出一致性导向的推理对齐（CORA），通过轻量级即插即用的一致性奖励模型将思维-答案语义一致性引入RLVR，并进一步引入混合奖励优势分割（HRAS）以稳定协调任务优化与一致性优化。在代表性多模态推理基准测试和主流LVLMs上的大量实验表明，CORA在提升任务性能的同时有效缓解了思维-答案不一致性，从而产生更可信的推理轨迹。