Vision-Language-Action (VLA) models achieve strong semantic generalization but often lack fine-grained modeling of world dynamics. Recent work explores video generation models as a foundation for world modeling, leading to unified World Action Models (WAMs) that jointly model visual dynamics and actions. We present MotuBrain, a unified multimodal generative model that jointly models video and action under a UniDiffuser formulation with a three-stream Mixture-of-Transformers architecture. A single model supports multiple inference modes, including policy learning, world modeling, video generation, inverse dynamics, and joint video-action prediction, while scaling to heterogeneous multimodal data such as video-only and cross-embodiment robot data. To improve real-world applicability, MotuBrain introduces a unified multiview representation, explicit language-action coupling, and an efficient inference stack, achieving over 50x speedup for real-time deployment.
翻译:视觉-语言-动作(VLA)模型实现了强大的语义泛化,但通常缺乏对世界动力学的细粒度建模。近期研究探索将视频生成模型作为世界建模的基础,从而催生了统一的世界动作模型(WAM),该模型可联合建模视觉动力学与动作。本文提出MotuBrain——一种统一的跨模态生成模型,在UniDiffuser框架下采用三流混合专家Transformer架构联合建模视频与动作。该单一模型支持包含策略学习、世界建模、视频生成、逆动力学及联合视频-动作预测在内的多种推理模式,同时可扩展至异构多模态数据(如纯视频数据和跨实体机器人数据)。为提升实际应用能力,MotuBrain引入了统一的多视角表征、显式语言-动作耦合机制及高效推理栈,在实时部署中实现了超过50倍的加速效果。