The ability to reuse collected data and transfer trained policies between robots could alleviate the burden of additional data collection and training. While existing approaches such as pretraining plus finetuning and co-training show promise, they do not generalize to robots unseen in training. Focusing on common robot arms with similar workspaces and 2-jaw grippers, we investigate the feasibility of zero-shot transfer. Through simulation studies on 8 manipulation tasks, we find that state-based Cartesian control policies can successfully zero-shot transfer to a target robot after accounting for forward dynamics. To address robot visual disparities for vision-based policies, we introduce Mirage, which uses "cross-painting"--masking out the unseen target robot and inpainting the seen source robot--during execution in real time so that it appears to the policy as if the trained source robot were performing the task. Despite its simplicity, our extensive simulation and physical experiments provide strong evidence that Mirage can successfully zero-shot transfer between different robot arms and grippers with only minimal performance degradation on a variety of manipulation tasks such as picking, stacking, and assembly, significantly outperforming a generalist policy. Project website: https://robot-mirage.github.io/
翻译:重复利用已收集数据并在机器人之间迁移已训练策略的能力,可减轻额外数据收集和训练负担。尽管预训练加微调、联合训练等现有方法展现出潜力,但它们无法泛化至训练中未见过的机器人。聚焦于具有相似工作空间和双夹爪抓持器的常见机械臂,我们探究了零样本迁移的可行性。通过对8个操作任务的仿真研究,我们发现:基于状态的空间笛卡尔控制策略在考虑前向动力学后,可成功实现向目标机器人的零样本迁移。为解决视觉策略中机器人视觉差异问题,我们提出Mirage方法——在执行过程中实时采用“交叉绘制”技术(即遮挡未见过的目标机器人并补全已知源机器人图像),从而使得策略感知到的场景如同训练所用源机器人在执行任务。尽管该方法简洁,但大量仿真与物理实验强有力地证明:Mirage可在不同机械臂和抓持器之间成功实现零样本迁移,且仅在抓取、堆叠、装配等多种操作任务中产生极轻微性能下降,显著优于通用策略。项目网站:https://robot-mirage.github.io/