Recently, methods for learning diverse skills to generate various behaviors without external rewards have been actively studied as a form of unsupervised reinforcement learning. However, most of the existing methods learn a finite number of discrete skills, and thus the variety of behaviors that can be exhibited with the learned skills is limited. In this paper, we propose a novel method for learning potentially an infinite number of different skills, which is named discovery of continuous skills on a sphere (DISCS). In DISCS, skills are learned by maximizing mutual information between skills and states, and each skill corresponds to a continuous value on a sphere. Because the representations of skills in DISCS are continuous, infinitely diverse skills could be learned. We examine existing methods and DISCS in the MuJoCo Ant robot control environments and show that DISCS can learn much more diverse skills than the other methods.

翻译：近年来，在无监督强化学习框架下，无需外部奖励即可学习多样化技能以生成各种行为的方法已成为研究热点。然而，现有方法大多局限于学习有限数量的离散技能，导致习得技能所能展现的行为多样性受限。本文提出一种名为"球面连续技能发现"（DISCS）的新方法，可实现潜在无限种不同技能的学习。DISCS通过最大化技能与状态之间的互信息来学习技能，每个技能对应球面上的连续数值。由于DISCS中技能的表示具有连续性，可学习到无限多样的技能。我们在MuJoCo蚂蚁机器人控制环境中对比评估了现有方法与DISCS，结果表明DISCS能够学习到比其他方法更多样化的技能。