Generalist robot manipulators need to learn a wide variety of manipulation skills across diverse environments. Current robot training pipelines rely on humans to provide kinesthetic demonstrations or to program simulation environments and to code up reward functions for reinforcement learning. Such human involvement is an important bottleneck towards scaling up robot learning across diverse tasks and environments. We propose Generation to Simulation (Gen2Sim), a method for scaling up robot skill learning in simulation by automating generation of 3D assets, task descriptions, task decompositions and reward functions using large pre-trained generative models of language and vision. We generate 3D assets for simulation by lifting open-world 2D object-centric images to 3D using image diffusion models and querying LLMs to determine plausible physics parameters. Given URDF files of generated and human-developed assets, we chain-of-thought prompt LLMs to map these to relevant task descriptions, temporal decompositions, and corresponding python reward functions for reinforcement learning. We show Gen2Sim succeeds in learning policies for diverse long horizon tasks, where reinforcement learning with non temporally decomposed reward functions fails. Gen2Sim provides a viable path for scaling up reinforcement learning for robot manipulators in simulation, both by diversifying and expanding task and environment development, and by facilitating the discovery of reinforcement-learned behaviors through temporal task decomposition in RL. Our work contributes hundreds of simulated assets, tasks and demonstrations, taking a step towards fully autonomous robotic manipulation skill acquisition in simulation.

翻译：通用型机器人操作器需在多样化环境中学习大量操作技能。当前机器人训练流程依赖人类提供动觉示教、编写仿真环境及为强化学习设计奖励函数，这种人工参与成为扩展机器人任务与环境多样性的关键瓶颈。我们提出"从生成到仿真"（Gen2Sim）方法，通过利用大规模预训练语言与视觉生成模型，自动化生成三维资产、任务描述、任务分解及奖励函数，从而实现模拟中机器人技能学习的规模化扩展。具体而言，我们采用图像扩散模型将开放世界的二维物体中心图像提升至三维空间以生成仿真资产，并通过查询大语言模型确定物理参数合理性。基于人类开发与程序生成的URDF文件，我们采用思维链提示引导大语言模型将其映射至相关任务描述、时序分解及对应的强化学习Python奖励函数。实验表明，Gen2Sim在多样化长序列任务中成功习得策略，而采用非时序分解奖励函数的强化学习方法则无法完成此类任务。该方法通过多样化任务与环境开发、以及基于时序任务分解加速强化学习行为发现，为扩大模拟中机器人操作器强化学习规模提供了可行路径。本工作贡献了数百个仿真资产、任务及示教数据，向实现完全自主的模拟机器人操作技能获取迈出了重要一步。