The most widely-adopted robot learning pipelines today learn skills from robot demonstrations or structured human data, which are expensive to collect and tied to specific embodiments. In contrast, unstructured human videos provide a scalable alternative. They contain diverse manipulation demonstrations across objects, scenes, and strategies, but are not directly connected to robot action. We propose LUCID, a two-stage framework that learns task intent from unstructured human videos drawn from internet-scale datasets and learns robot control in massively-parallel simulation. The intent model predicts short-horizon intent (what should happen next in the scene) from the current observation in closed loop. An embodiment-specific sensorimotor policy converts this intent into robot actions. The intent interface is shared across controllers, so the same intent model can be applied to different embodiments, from our primary dexterous hand to a parallel-jaw gripper. We evaluate LUCID on five real-world manipulation tasks: stirring, wiping, and binning supervised by only internet video, with zero-shot transfer to novel scenes and object instances; and push-T and cable routing supervised by 1 hr each of self-collected smartphone video. Project page: https://lucid-robot.github.io/.

翻译：当前最广泛采用的机器人学习流程通常依赖于机器人演示或结构化人类数据来学习技能，这些数据不仅采集成本高昂，而且与特定具身形式紧密绑定。相比之下，非结构化人类视频提供了一种可扩展的替代方案——它们包含了跨物体、场景和策略的多样化操作演示，但并未直接关联到机器人动作。我们提出LUCID，这是一个两阶段框架：首先从互联网规模数据集中抽取的非结构化人类视频中学习任务意图，然后在大规模并行仿真环境中学习机器人控制。该意图模型通过闭环方式根据当前观测预测短期意图（即场景中接下来应发生的事件）。具身特异性感觉运动策略将该意图转化为机器人动作。由于意图接口在多个控制器之间共享，相同的意图模型可适用于不同具身形式，从我们主要的灵巧手到平行夹爪。我们在五项真实世界操作任务上评估了LUCID：仅通过互联网视频监督即可完成的搅拌、擦拭和分拣任务（零样本迁移至新场景及物体实例）；以及分别由1小时自采集智能手机视频监督的推T和线缆布线任务。项目页面：https://lucid-robot.github.io/。