We present RoboGen, a generative robotic agent that automatically learns diverse robotic skills at scale via generative simulation. RoboGen leverages the latest advancements in foundation and generative models. Instead of directly using or adapting these models to produce policies or low-level actions, we advocate for a generative scheme, which uses these models to automatically generate diversified tasks, scenes, and training supervisions, thereby scaling up robotic skill learning with minimal human supervision. Our approach equips a robotic agent with a self-guided propose-generate-learn cycle: the agent first proposes interesting tasks and skills to develop, and then generates corresponding simulation environments by populating pertinent objects and assets with proper spatial configurations. Afterwards, the agent decomposes the proposed high-level task into sub-tasks, selects the optimal learning approach (reinforcement learning, motion planning, or trajectory optimization), generates required training supervision, and then learns policies to acquire the proposed skill. Our work attempts to extract the extensive and versatile knowledge embedded in large-scale models and transfer them to the field of robotics. Our fully generative pipeline can be queried repeatedly, producing an endless stream of skill demonstrations associated with diverse tasks and environments.

翻译：本文提出RoboGen，一种通过生成式仿真自动规模化学习多样化机器人技能的生成式机器人智能体。RoboGen充分利用基础模型与生成模型的最新进展。我们主张采用生成式范式——利用这些模型自动生成多样化的任务、场景与训练监督信号，而非直接使用或调整这些模型来输出策略或底层动作，从而以最少的人类监督实现机器人技能学习的规模化扩展。我们的方法赋予机器人智能体自我引导的"提议-生成-学习"循环：智能体首先提议待开发的有趣任务与技能，随后通过配置相关物体与资产的空间布局生成对应的仿真环境。接着，智能体将提议的高层任务分解为子任务，选择最优学习方法（强化学习、运动规划或轨迹优化），生成必要的训练监督信号，进而通过策略学习掌握所提议的技能。本研究尝试提取大规模模型中蕴含的广泛而通用的知识，并将其迁移至机器人领域。我们的全生成式流程可重复调用，持续产生与多样化任务及环境相关联的技能演示流。