We propose X-WAM, a Unified 4D World Model that unifies real-time robotic action execution and high-fidelity 4D world synthesis (video + 3D reconstruction) in a single framework, addressing the critical limitations of prior unified world models (e.g., UWM) that only model 2D pixel-space and fail to balance action efficiency and world modeling quality. To leverage the strong visual priors of pretrained video diffusion models, X-WAM imagines the future world by predicting multi-view RGB-D videos, and obtains spatial information efficiently through a lightweight structural adaptation: replicating the final few blocks of the pretrained Diffusion Transformer into a dedicated depth prediction branch for the reconstruction of future spatial information. Moreover, we propose Asynchronous Noise Sampling (ANS) to jointly optimize generation quality and action decoding efficiency. ANS applies a specialized asynchronous denoising schedule during inference, which rapidly decodes actions with fewer steps to enable efficient real-time execution, while dedicating the full sequence of steps to generate high-fidelity video. Rather than entirely decoupling the timesteps during training, ANS samples from their joint distribution to align with the inference distribution. Pretrained on over 5,800 hours of robotic data, X-WAM achieves 79.2% and 90.7% average success rate on RoboCasa and RoboTwin 2.0 benchmarks, while producing high-fidelity 4D reconstruction and generation surpassing existing methods in both visual and geometric metrics.
翻译:我们提出X-WAM,一种统一4D世界模型,将实时机器人动作执行与高保真4D世界合成(视频+3D重建)统一于单一框架中,解决了先前统一世界模型(如UWM)仅建模2D像素空间、且无法平衡动作效率与世界建模质量的关键局限性。为利用预训练视频扩散模型强大的视觉先验,X-WAM通过预测多视角RGB-D视频来想象未来世界,并通过轻量级结构适配高效获取空间信息:将预训练扩散变换器的最后几个模块复制到专用深度预测分支中,用于重建未来空间信息。此外,我们提出异步噪声采样(ANS)以联合优化生成质量与动作解码效率。ANS在推理过程中采用专门的异步去噪调度,以较少步骤快速解码动作从而实现高效实时执行,同时将完整步骤序列用于生成高保真视频。ANS并非在训练时完全解耦时间步长,而是从其联合分布中采样以与推理分布对齐。在超过5800小时机器人数据上预训练的X-WAM,在RoboCasa和RoboTwin 2.0基准测试中分别达到79.2%和90.7%的平均成功率,同时产生的高保真4D重建与生成在视觉和几何指标上均超越现有方法。