We present RoboGen, a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. RoboGen leverages the latest advancements in foundation and generative models. Instead of directly using or adapting these models to produce policies or low-level actions, we advocate for a generative scheme, which uses these models to automatically generate diversified tasks, scenes, and training supervisions, thereby scaling up robotic skill learning with minimal human supervision. Our approach equips a robotic agent with a self-guided propose-generate-learn cycle: the agent first proposes interesting tasks and skills to develop, and then generates corresponding simulation environments by populating pertinent objects and assets with proper spatial configurations. Afterwards, the agent decomposes the proposed high-level task into sub-tasks, selects the optimal learning approach (reinforcement learning, motion planning, or trajectory optimization), generates required training supervision, and then learns policies to acquire the proposed skill. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics. Our fully generative pipeline can be queried repeatedly, producing an endless stream of skill demonstrations associated with diverse tasks and environments.

翻译：我们提出RoboGen，一种通过生成式仿真自动学习多样化机器人技能的生成式机器人智能体。RoboGen利用基础模型与生成模型的最新进展。我们不直接使用或调整这些模型来生成策略或低级动作，而是倡导一种生成式方案：利用这些模型自动生成多样化的任务、场景及训练监督信号，从而在最小人工监督下扩展机器人技能学习。该方法赋予机器人智能体一个自我引导的“提出-生成-学习”循环：智能体首先提出待开发的有趣任务与技能，随后通过填充具有恰当空间配置的相关物体与资源生成对应的仿真环境。接着，智能体将所提议的高级任务分解为子任务，选择最优学习方式（强化学习、运动规划或轨迹优化），生成所需的训练监督，并学习策略以获取提议的技能。我们的工作试图提取大规模模型中蕴含的广泛而多用途知识，并将其迁移至机器人领域。这一完全生成式流水线可被反复调用，从而持续产生与多样化任务和场景关联的技能演示数据流。