Effective robotic manipulation requires policies that can anticipate physical outcomes and adapt to real-world environments. Effective robotic manipulation requires policies that can anticipate physical outcomes and adapt to real-world environments. In this work, we introduce a unified framework, World-Model-Driven Diffusion Policy with Online Adaptive Learning (AdaWorldPolicy) to enhance robotic manipulation under dynamic conditions with minimal human involvement. Our core insight is that world models provide strong supervision signals, enabling online adaptive learning in dynamic environments, which can be complemented by force-torque feedback to mitigate dynamic force shifts. Our AdaWorldPolicy integrates a world model, an action expert, and a force predictor-all implemented as interconnected Flow Matching Diffusion Transformers (DiT). They are interconnected via the multi-modal self-attention layers, enabling deep feature exchange for joint learning while preserving their distinct modularity characteristics. We further propose a novel Online Adaptive Learning (AdaOL) strategy that dynamically switches between an Action Generation mode and a Future Imagination mode to drive reactive updates across all three modules. This creates a powerful closed-loop mechanism that adapts to both visual and physical domain shifts with minimal overhead. Across a suite of simulated and real-robot benchmarks, our AdaWorldPolicy achieves state-of-the-art performance, with dynamical adaptive capacity to out-of-distribution scenarios.

翻译：有效的机器人操作需要能够预测物理结果并适应现实世界环境的策略。在这项工作中，我们提出了一个统一框架——具备在线自适应学习能力的世界模型驱动扩散策略（AdaWorldPolicy），旨在以最少的人力介入，增强动态条件下的机器人操作能力。我们的核心见解是，世界模型能够提供强监督信号，实现在动态环境中的在线自适应学习，并可辅以力-力矩反馈来缓解动态力的偏移。我们的AdaWorldPolicy集成了一个世界模型、一个动作专家和一个力预测器——三者均以相互关联的流匹配扩散Transformer（DiT）实现。它们通过多模态自注意力层相互连接，实现了用于联合学习的深度特征交换，同时保留了各自独特的模块化特性。我们进一步提出了一种新颖的在线自适应学习（AdaOL）策略，该策略在动作生成模式和未来想象模式之间动态切换，以驱动所有三个模块的响应式更新。这形成了一个强大的闭环机制，能够以最小的开销适应视觉和物理领域的偏移。在一系列仿真和真实机器人基准测试中，我们的AdaWorldPolicy实现了最先进的性能，并具备应对分布外场景的动态自适应能力。