One of the key capabilities of intelligent agents is the ability to discover useful skills without external supervision. However, the current unsupervised skill discovery methods are often limited to acquiring simple, easy-to-learn skills due to the lack of incentives to discover more complex, challenging behaviors. We introduce a novel unsupervised skill discovery method, Controllability-aware Skill Discovery (CSD), which actively seeks complex, hard-to-control skills without supervision. The key component of CSD is a controllability-aware distance function, which assigns larger values to state transitions that are harder to achieve with the current skills. Combined with distance-maximizing skill discovery, CSD progressively learns more challenging skills over the course of training as our jointly trained distance function reduces rewards for easy-to-achieve skills. Our experimental results in six robotic manipulation and locomotion environments demonstrate that CSD can discover diverse complex skills including object manipulation and locomotion skills with no supervision, significantly outperforming prior unsupervised skill discovery methods. Videos and code are available at https://seohong.me/projects/csd/

翻译：智能体的关键能力之一是在没有外部监督的情况下发现有用技能。然而，当前的無监督技能发现方法由于缺乏激励去发现更复杂、更具挑战性的行为，往往局限于获取简单、易学的技能。我们提出了一种新颖的无监督技能发现方法——可控性感知技能发现（CSD），该方法在无监督条件下主动寻求复杂、难控制的技能。CSD的核心是一个可控性感知距离函数，该函数为当前技能难以实现的状态转移赋予更大的值。结合最大化距离的技能发现，随着联合训练的距离函数降低易实现技能的奖励，CSD在训练过程中逐步学习更具挑战性的技能。我们在六个机器人操作和运动环境中的实验结果表明，CSD能够在无监督条件下发现包括物体操作和运动技能在内的多样化复杂技能，其性能显著优于先前的无监督技能发现方法。视频和代码可在 https://seohong.me/projects/csd/ 获取。