The standard post-training recipe for large multimodal models (LMMs) applies supervised fine-tuning (SFT) on curated demonstrations followed by reinforcement learning with verifiable rewards (RLVR). However, SFT introduces distributional drift that neither preserves the model's original capabilities nor faithfully matches the supervision distribution. This problem is further amplified in multimodal reasoning, where perception errors and reasoning failures follow distinct drift patterns that compound during subsequent RL. We introduce PRISM, a three-stage pipeline that mitigates this drift by inserting an explicit distribution-alignment stage between SFT and RLVR. Building on the principle of on-policy distillation (OPD), PRISM casts alignment as a black-box, response-level adversarial game between the policy and a Mixture-of-Experts (MoE) discriminator with dedicated perception and reasoning experts, providing disentangled corrective signals that steer the policy toward the supervision distribution without requiring access to teacher logits. While 1.26M public demonstrations suffice for broad SFT initialization, distribution alignment demands higher-fidelity supervision; we therefore curate 113K additional demonstrations from Gemini 3 Flash, featuring dense visual grounding and step-by-step reasoning on the hardest unsolved problems. Experiments on Qwen3-VL show that PRISM consistently improves downstream RLVR performance across multiple RL algorithms (GRPO, DAPO, GSPO) and diverse multimodal benchmarks, improving average accuracy by +4.4 and +6.0 points over the SFT-to-RLVR baseline on 4B and 8B, respectively. Our code, data, and model checkpoints are publicly available at https://github.com/XIAO4579/PRISM.

翻译：[translated abstract in Chinese] 大型多模态模型（LMMs）的标准后训练流程，首先在精选演示数据上进行监督微调（SFT），随后采用可验证奖励的强化学习（RLVR）。然而，监督微调引入了分布偏移，既未能保持模型的原始能力，也未能完全匹配监督分布。该问题在多模态推理场景中进一步加剧——感知误差与推理失败遵循不同的偏移模式，并在后续强化学习过程中叠加累积。我们提出PRISM，一种通过在三阶段流程中插入显式分布对齐阶段来缓解该偏移的三阶段流水线。基于在线策略蒸馏（OPD）原理，PRISM将对齐过程建模为策略模型与具有专用感知与推理专家的混合专家（MoE）判别器之间的黑盒、响应级对抗博弈，提供解耦的校正信号，引导策略模型逼近监督分布，而无需访问教师逻辑值。虽然仅需12.6万条公开演示数据即可实现广泛的SFT初始化，但分布对齐要求更高保真度的监督信号；因此我们从Gemini 3 Flash额外筛选11.3万条演示数据，涵盖密集视觉锚定与对最困难未解问题的逐步推理。在Qwen3-VL上的实验表明，PRISM在多种强化学习算法（GRPO、DAPO、GSPO）与多样化多模态基准上持续提升下游RLVR性能，在4B与8B参数规模下分别比SFT-to-RLVR基线提升平均准确率4.4和6.0个百分点。我们的代码、数据与模型检查点已开源至https://github.com/XIAO4579/PRISM。