We present RoboGen, a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. RoboGen leverages the latest advancements in foundation and generative models. Instead of directly using or adapting these models to produce policies or low-level actions, we advocate for a generative scheme, which uses these models to automatically generate diversified tasks, scenes, and training supervisions, thereby scaling up robotic skill learning with minimal human supervision. Our approach equips a robotic agent with a self-guided propose-generate-learn cycle: the agent first proposes interesting tasks and skills to develop, and then generates corresponding simulation environments by populating pertinent objects and assets with proper spatial configurations. Afterwards, the agent decomposes the proposed high-level task into sub-tasks, selects the optimal learning approach (reinforcement learning, motion planning, or trajectory optimization), generates required training supervision, and then learns policies to acquire the proposed skill. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics. Our fully generative pipeline can be queried repeatedly, producing an endless stream of skill demonstrations associated with diverse tasks and environments.

翻译：我们提出了RoboGen，一种通过生成式仿真自动大规模学习多样化机器人技能的生成式机器人智能体。RoboGen利用了基础模型与生成模型的最新进展。我们不直接使用或调整这些模型来生成策略或低级动作，而是倡导一种生成式方案，即利用这些模型自动生成多样化的任务、场景和训练监督信号，从而在最小化人工监督的条件下扩展机器人技能学习。我们的方法赋予机器人智能体一个自主的“提议-生成-学习”循环：智能体首先提出需要发展的有趣任务与技能，随后通过配置具有适当空间布局的相关物体与资源，生成对应的仿真环境。之后，智能体将提出的高层任务分解为子任务，选择最优学习方式（强化学习、运动规划或轨迹优化），生成所需的训练监督信号，最后学习策略以掌握所提议的技能。本文尝试提取大规模模型中蕴含的广泛而丰富的知识，并将其迁移至机器人领域。我们的全生成式流程可被反复调用，产生与多样化任务和环境相关的无限技能演示流。