Wrist-view observations are crucial for VLA models as they capture fine-grained hand-object interactions that directly enhance manipulation performance. Yet large-scale datasets rarely include such recordings, resulting in a substantial gap between abundant anchor views and scarce wrist views. Existing world models cannot bridge this gap, as they require a wrist-view first frame and thus fail to generate wrist-view videos from anchor views alone. Amid this gap, recent visual geometry models such as VGGT emerge with geometric and cross-view priors that make it possible to address extreme viewpoint shifts. Inspired by these insights, we propose WristWorld, the first 4D world model that generates wrist-view videos solely from anchor views. WristWorld operates in two stages: (i) Reconstruction, which extends VGGT and incorporates our Spatial Projection Consistency (SPC) Loss to estimate geometrically consistent wrist-view poses and 4D point clouds; (ii) Generation, which employs our video generation model to synthesize temporally coherent wrist-view videos from the reconstructed perspective. Experiments on Droid, Calvin, and Franka Panda demonstrate state-of-the-art video generation with superior spatial consistency, while also improving VLA performance, raising the average task completion length on Calvin by 3.81% and closing 42.4% of the anchor-wrist view gap.
翻译:腕部视角观测对于VLA模型至关重要,其能捕捉细粒度的手-物交互,直接提升操控性能。然而大规模数据集鲜少包含此类记录,导致丰富的锚定视角与稀缺的腕部视角之间存在显著差距。现有世界模型无法弥合此差距,因其需要腕部视角的首帧图像,从而无法仅从锚定视角生成腕部视角视频。在此背景下,近期出现的VGGT等视觉几何模型凭借几何与跨视角先验知识,为处理极端视角偏移提供了可能。受此启发,我们提出WristWorld——首个仅通过锚定视角即可生成腕部视角视频的4D世界模型。WristWorld分两阶段运行:(一)重建阶段:扩展VGGT框架并引入空间投影一致性损失函数,以估算几何一致的腕部视角位姿与4D点云;(二)生成阶段:采用视频生成模型从重建视角合成时序连贯的腕部视角视频。在Droid、Calvin和Franka Panda平台上的实验表明,本方法实现了具有卓越空间一致性的前沿视频生成性能,同时将VLA任务平均完成长度在Calvin数据集上提升3.81%,并弥合了42.4%的锚定-腕部视角差距。