Eye-in-hand cameras have shown promise in enabling greater sample efficiency and generalization in vision-based robotic manipulation. However, for robotic imitation, it is still expensive to have a human teleoperator collect large amounts of expert demonstrations with a real robot. Videos of humans performing tasks, on the other hand, are much cheaper to collect since they eliminate the need for expertise in robotic teleoperation and can be quickly captured in a wide range of scenarios. Therefore, human video demonstrations are a promising data source for learning generalizable robotic manipulation policies at scale. In this work, we augment narrow robotic imitation datasets with broad unlabeled human video demonstrations to greatly enhance the generalization of eye-in-hand visuomotor policies. Although a clear visual domain gap exists between human and robot data, our framework does not need to employ any explicit domain adaptation method, as we leverage the partial observability of eye-in-hand cameras as well as a simple fixed image masking scheme. On a suite of eight real-world tasks involving both 3-DoF and 6-DoF robot arm control, our method improves the success rates of eye-in-hand manipulation policies by 58% (absolute) on average, enabling robots to generalize to both new environment configurations and new tasks that are unseen in the robot demonstration data. See video results at https://giving-robots-a-hand.github.io/ .

翻译：手眼相机在提升基于视觉的机器人操作中的样本效率和泛化能力方面展现出潜力。然而，对于机器人模仿学习而言，通过人类遥操作在真实机器人上收集大量专家演示仍成本高昂。相比之下，人类执行任务的视频收集成本低得多，既无需专业遥操作技能，又能快速覆盖多样化场景。因此，人类视频演示成为规模化学习可泛化机器人操作策略的重要数据源。本研究通过将狭窄的机器人模仿数据集与广泛的无标注人类视频演示相结合，显著增强了手眼视觉运动策略的泛化能力。尽管人类与机器人数据存在明显的视觉领域差异，我们的框架无需采用任何显式领域自适应方法，而是利用手眼相机的局部可观测性及其简单的固定图像掩码方案。在涵盖3自由度和6自由度机械臂控制的8项真实世界任务中，所提方法将手眼操作策略的成功率平均提升58%（绝对值），使机器人能够泛化到未出现在机器人演示数据中的新环境配置与新任务。视频结果参见https://giving-robots-a-hand.github.io/。