Many recent advances in robotic manipulation have come through imitation learning, yet these rely largely on mimicking a particularly hard-to-acquire form of demonstrations: those collected on the same robot in the same room with the same objects as the trained policy must handle at test time. In contrast, large pre-recorded human video datasets demonstrating manipulation skills in-the-wild already exist, which contain valuable information for robots. Is it possible to distill a repository of useful robotic skill policies out of such data without any additional requirements on robot-specific demonstrations or exploration? We present the first such system ZeroMimic, that generates immediately deployable image goal-conditioned skill policies for several common categories of manipulation tasks (opening, closing, pouring, pick&place, cutting, and stirring) each capable of acting upon diverse objects and across diverse unseen task setups. ZeroMimic is carefully designed to exploit recent advances in semantic and geometric visual understanding of human videos, together with modern grasp affordance detectors and imitation policy classes. After training ZeroMimic on the popular EpicKitchens dataset of ego-centric human videos, we evaluate its out-of-the-box performance in varied real-world and simulated kitchen settings with two different robot embodiments, demonstrating its impressive abilities to handle these varied tasks. To enable plug-and-play reuse of ZeroMimic policies on other task setups and robots, we release software and policy checkpoints of our skill policies.

翻译：机器人操作领域的许多近期进展都通过模仿学习实现，但这些方法主要依赖于模仿一种特别难以获取的演示形式：即在测试时策略必须处理的相同机器人、相同房间、相同物体上收集的演示。相比之下，大量预先录制的人类视频数据集已经存在，它们展示了真实环境中的操作技能，其中包含了对于机器人有价值的信息。是否有可能仅从这类数据中蒸馏出一系列有用的机器人技能策略，而无需任何额外的、针对特定机器人的演示或探索要求？我们提出了首个此类系统 ZeroMimic，它能生成可立即部署的图像目标条件技能策略，适用于多个常见类别的操作任务（打开、关闭、倾倒、拾取放置、切割和搅拌），每个策略都能够处理多样化的物体，并适应多样化的、未见过的任务场景。ZeroMimic 经过精心设计，以利用近期在人类视频的语义和几何视觉理解方面的进展，结合现代抓取可供性检测器和模仿策略类别。在流行的人类第一人称视角视频数据集 EpicKitchens 上训练 ZeroMimic 后，我们在两种不同机器人形态的多样化真实世界和模拟厨房环境中评估其开箱即用的性能，展示了其处理这些多样化任务的出色能力。为了支持在其他任务场景和机器人上即插即用地复用 ZeroMimic 策略，我们发布了我们技能策略的软件和策略检查点。