Despite the sustained scaling on model capacity and data acquisition, Vision-Language-Action (VLA) models remain brittle in contact-rich and dynamic manipulation tasks, where minor execution deviations can compound into failures. While reinforcement learning (RL) offers a principled path to robustness, on-policy RL in the physical world is constrained by safety risk, hardware cost, and environment reset. To bridge this gap, we present RISE, a scalable framework of robotic reinforcement learning via imagination. At its core is a Compositional World Model that (i) predicts multi-view future via a controllable dynamics model, and (ii) evaluates imagined outcomes with a progress value model, producing informative advantages for the policy improvement. Such compositional design allows state and value to be tailored by best-suited yet distinct architectures and objectives. These components are integrated into a closed-loop self-improving pipeline that continuously generates imaginary rollouts, estimates advantages, and updates the policy in imaginary space without costly physical interaction. Across three challenging real-world tasks, RISE yields significant improvement over prior art, with more than +35% absolute performance increase in dynamic brick sorting, +45% for backpack packing, and +35% for box closing, respectively.

翻译：摘要：尽管在模型容量和数据获取方面持续扩展，视觉-语言-动作（VLA）模型在接触密集和动态操作任务中仍然脆弱，其中微小的执行偏差可能累积为失败。虽然强化学习（RL）为鲁棒性提供了原则性路径，但物理世界中的在策略RL受限于安全风险、硬件成本和环境重置。为弥补这一差距，我们提出RISE，一种通过想象进行机器人强化学习的可扩展框架。其核心是组合世界模型，该模型（i）通过可控动力学模型预测多视图未来，（ii）用进度价值模型评估想象结果，为策略改进提供信息优势。这种组合设计允许状态和价值由最适合但不同的架构和目标定制。这些组件集成到一个闭环自改进流水线中，该流水线持续生成想象轨迹、估计优势、并在想象空间中更新策略，无需昂贵的物理交互。在三个具有挑战性的现实任务中，RISE相比先前技术取得了显著提升，在动态砖块分类中绝对性能提升超过+35%，背包打包提升+45%，盒子关闭提升+35%。