Learning transferable knowledge from unlabeled video data and applying it in new environments is a fundamental capability of intelligent agents. This work presents VideoWorld 2, which extends VideoWorld and offers the first investigation into learning transferable knowledge directly from raw real-world videos. At its core, VideoWorld 2 introduces a dynamic-enhanced Latent Dynamics Model (dLDM) that decouples action dynamics from visual appearance: a pretrained video diffusion model handles visual appearance modeling, enabling the dLDM to learn latent codes that focus on compact and meaningful task-related dynamics. These latent codes are then modeled autoregressively to learn task policies and support long-horizon reasoning. We evaluate VideoWorld 2 on challenging real-world handcraft making tasks, where prior video generation and latent-dynamics models struggle to operate reliably. Remarkably, VideoWorld 2 achieves up to 70% improvement in task success rate and produces coherent long execution videos. In robotics, we show that VideoWorld 2 can acquire effective manipulation knowledge from the Open-X dataset, which substantially improves task performance on CALVIN. This study reveals the potential of learning transferable world knowledge directly from raw videos, with all code, data, and models to be open-sourced for further research.

翻译：从无标签视频数据中学习可迁移知识并将其应用于新环境，是智能体的一项基本能力。本文提出VideoWorld 2，它扩展了VideoWorld，并首次探究了直接从原始真实世界视频中学习可迁移知识的方法。其核心是引入了一个动态增强的隐式动态模型（dLDM），该模型将动作动态与视觉外观解耦：一个预训练的视频扩散模型负责视觉外观建模，使得dLDM能够学习专注于紧凑且有意义任务相关动态的隐式编码。随后，这些隐式编码通过自回归方式进行建模，以学习任务策略并支持长时程推理。我们在具有挑战性的真实世界手工艺制作任务上评估VideoWorld 2，而先前的视频生成和隐式动态模型在此类任务中难以可靠运行。值得注意的是，VideoWorld 2在任务成功率上实现了高达70%的提升，并生成了连贯的长时程执行视频。在机器人领域，我们展示了VideoWorld 2能够从Open-X数据集中获取有效的操作知识，这显著提升了在CALVIN基准上的任务性能。本研究揭示了直接从原始视频中学习可迁移世界知识的潜力，所有代码、数据和模型都将开源以供进一步研究。