Imitation learning from human demonstrations is an effective means to teach robots manipulation skills. But data acquisition is a major bottleneck in applying this paradigm more broadly, due to the amount of cost and human effort involved. There has been significant interest in imitation learning for bimanual dexterous robots, like humanoids. Unfortunately, data collection is even more challenging here due to the challenges of simultaneously controlling multiple arms and multi-fingered hands. Automated data generation in simulation is a compelling, scalable alternative to fuel this need for data. To this end, we introduce DexMimicGen, a large-scale automated data generation system that synthesizes trajectories from a handful of human demonstrations for humanoid robots with dexterous hands. We present a collection of simulation environments in the setting of bimanual dexterous manipulation, spanning a range of manipulation behaviors and different requirements for coordination among the two arms. We generate 21K demos across these tasks from just 60 source human demos and study the effect of several data generation and policy learning decisions on agent performance. Finally, we present a real-to-sim-to-real pipeline and deploy it on a real-world humanoid can sorting task. Videos and more are at https://dexmimicgen.github.io/

翻译：基于人类演示的模仿学习是教授机器人操作技能的有效手段。然而，由于涉及高昂的成本和大量人力投入，数据获取成为更广泛应用这一范式的主要瓶颈。对于类人机器人等双手灵巧机器人，模仿学习已引起广泛关注。不幸的是，由于需要同时控制多个手臂和多指灵巧手，数据收集在此领域更具挑战性。仿真环境中的自动化数据生成作为一种可扩展的替代方案，为满足数据需求提供了极具吸引力的解决方案。为此，我们提出了DexMimicGen，一个大规模自动化数据生成系统，能够基于少量人类演示为配备灵巧手的类人机器人合成轨迹。我们构建了一套双手灵巧操作场景下的仿真环境集合，涵盖多种操作行为及双臂间不同协调需求的任务。仅从60个人源演示出发，我们在这些任务中生成了21K个演示样本，并研究了多种数据生成与策略学习决策对智能体性能的影响。最后，我们提出了一套从真实到仿真再到真实的部署流程，并将其应用于现实世界的类人机器人易拉罐分拣任务。视频及更多内容请访问：https://dexmimicgen.github.io/