Embodied agents need to predict the future consequences of candidate actions in order to plan effectively before execution. Existing visual dynamics models learn by reconstructing future visual states or rolling out dense latent representations, which spreads learning capacity across visually salient but planning-irrelevant content rather than the action-conditioned changes that drive manipulation outcomes. We propose CAPE, a Contrastive Action-conditioned Parallel Encoding framework that learns visual dynamics by distinguishing the future outcomes induced by different action sequences. Given an initial observation and a candidate action sequence, CAPE decodes the full future latent trajectory in a single forward pass and is trained with a Goal-Convergent Contrastive Objective that aligns predictions corresponding to the same future outcome while separating those corresponding to different outcomes. On real-world DROID and zero-shot transfer to RoboCasa, CAPE substantially outperforms prior baselines on future-state retrieval, offline action matching, and closed-loop planning, while notably reducing planning-time inference cost at long prediction horizons.
翻译:具身智能体需要在执行前有效规划,以预测候选行动的未来后果。现有视觉动力学模型通过重建未来视觉状态或展开密集的潜在表征进行学习,这会将学习能力分散到视觉显著但与规划无关的内容上,而非驱动操作结果的行动条件变化中。我们提出CAPE,一种对比性行动条件并行编码框架,通过区分不同行动序列引发的未来结果来学习视觉动力学。给定初始观测和候选行动序列,CAPE通过单次前向传递解码完整未来潜在轨迹,并采用目标收敛对比目标进行训练:该目标对齐对应相同未来结果的预测,同时分离对应不同结果的预测。在真实世界数据集DROID及零样本迁移至RoboCasa的场景中,CAPE在未来状态检索、离线行动匹配和闭环规划方面显著超越先前基线模型,同时在长预测范围内显著降低了规划时的推理成本。