Can we turn a video prediction model into a robot policy? Videos, including those of humans or teleoperated robots, capture rich physical interactions. However, most of them lack labeled actions, which limits their use in robot learning. We present Video Prediction for Robot Actions (ViPRA), a simple pretraining-finetuning framework that learns continuous robot control from these actionless videos. Instead of directly predicting actions, we train a video-language model to predict both future visual observations and motion-centric latent actions, which serve as intermediate representations of scene dynamics. We train these latent actions using perceptual losses and optical flow consistency to ensure they reflect physically grounded behavior. For downstream control, we introduce a chunked flow matching decoder that maps latent actions to robot-specific continuous action sequences, using only 100 to 200 teleoperated demonstrations. This approach avoids expensive action annotation, supports generalization across embodiments, and enables smooth, high-frequency continuous control upto 22 Hz via chunked action decoding. Unlike prior latent action works that treat pretraining as autoregressive policy learning, ViPRA explicitly models both what changes and how. Our method outperforms strong baselines, with a 16% gain on the SIMPLER benchmark and a 13% improvement across real world manipulation tasks. We have released models and code at https://vipra-project.github.io

翻译：能否将视频预测模型转化为机器人策略？视频，包括人类或远程操作机器人的视频，捕捉了丰富的物理交互信息。然而，大多数视频缺乏标注动作，这限制了它们在机器人学习中的应用。我们提出面向机器人动作的视频预测（ViPRA）——一种简单的预训练-微调框架，能够从这些无动作视频中学习连续的机器人控制。不同于直接预测动作，我们训练一个视频语言模型来同时预测未来视觉观察和以运动为中心的潜在动作，这些潜在动作作为场景动态的中间表征。我们使用感知损失和光流一致性来训练这些潜在动作，确保其反映物理基础行为。在下游控制中，我们引入一种分块流匹配解码器，仅需100至200条遥操作演示数据，即可将潜在动作映射为机器人特定的连续动作序列。该方法避免了昂贵的动作标注，支持跨本体泛化，并通过分块动作解码实现高达22Hz的平滑、高频连续控制。与将预训练视为自回归策略学习的先前潜在动作工作不同，ViPRA显式建模了“变化内容”与“变化方式”。我们的方法在SIMPLER基准上取得16%的性能提升，在真实世界操作任务中实现13%的改进，显著优于强基线。模型和代码已在https://vipra-project.github.io开源。