Large-scale robot datasets have facilitated the learning of a wide range of robot manipulation skills, but these datasets remain difficult to collect and scale further, owing to the intractable amount of human time, effort, and cost required. Simulation and synthetic data generation have proven to be an effective alternative to fuel this need for data, especially with the advent of recent work showing that such synthetic datasets can dramatically reduce real-world data requirements and facilitate generalization to novel scenarios unseen in real-world demonstrations. However, this paradigm has been limited to rigid-body tasks, which are easy to simulate. Deformable object manipulation encompasses a large portion of real-world manipulation and remains a crucial gap to address towards increasing adoption of the synthetic simulation data paradigm. In this paper, we introduce SoftMimicGen, an automated data generation pipeline for deformable object manipulation tasks. We introduce a suite of high-fidelity simulation environments that encompasses a wide range of deformable objects (stuffed animal, rope, tissue, towel) and manipulation behaviors (high-precision threading, dynamic whipping, folding, pick-and-place), across four robot embodiments: a single-arm manipulator, bimanual arms, a humanoid, and a surgical robot. We apply SoftMimicGen to generate datasets across the task suite, train high-performing policies from the data, and systematically analyze the data generation system. Project website: \href{https://softmimicgen.github.io}{softmimicgen.github.io}.
翻译:大规模机器人数据集已促进了多种机器人操作技能的学习,但由于所需人力、时间和成本难以估量,这些数据集仍难以收集并进一步扩展。仿真与合成数据生成已被证明是满足数据需求的有效替代方案,尤其是近期研究表明,此类合成数据集可显著减少对真实世界数据的需求,并有助于泛化到真实演示中未见的新场景。然而,这一范式此前仅局限于易于仿真的刚体任务。可变形物体操作涵盖了大量真实世界中的操作任务,仍是推动合成仿真数据范式广泛应用的关键缺口。本文提出了SoftMimicGen——一个面向可变形物体操作任务的自动化数据生成流水线。我们引入了一套高保真仿真环境,涵盖多种可变形物体(毛绒玩具、绳索、纸巾、毛巾)和操作行为(高精度穿线、动态甩动、折叠、抓取和放置),并涉及四种机器人实体:单臂机械手、双臂机械手、人形机器人和手术机器人。我们应用SoftMimicGen在整套任务中生成数据集,从数据中训练高性能策略,并系统性地分析了该数据生成系统。项目网站:\href{https://softmimicgen.github.io}{softmimicgen.github.io}。