What is a useful skill hierarchy for an autonomous agent? We propose an answer based on the graphical structure of an agent's interaction with its environment. Our approach uses hierarchical graph partitioning to expose the structure of the graph at varying timescales, producing a skill hierarchy with multiple levels of abstraction. At each level of the hierarchy, skills move the agent between regions of the state space that are well connected within themselves but weakly connected to each other. We illustrate the utility of the proposed skill hierarchy in a wide variety of domains in the context of reinforcement learning.

翻译：自主智能体应具备何种有用的技能层级结构？本文基于智能体与环境交互的图结构提出一种解决方案。该方法采用分层图划分技术，在不同时间尺度上揭示图的拓扑结构，从而生成具有多层次抽象能力的技能层级体系。在该层级的每一层，技能引导智能体在状态空间内强连通且弱耦合的区域间移动。我们通过强化学习框架在多种应用场景中验证了所提技能层级结构的有效性。