Imitation learning is an approach in which an agent learns how to execute a task by trying to mimic how one or more teachers perform it. This learning approach offers a compromise between the time it takes to learn a new task and the effort needed to collect teacher samples for the agent. It achieves this by balancing learning from the teacher, who has some information on how to perform the task, and deviating from their examples when necessary, such as states not present in the teacher samples. Consequently, the field of imitation learning has received much attention from researchers in recent years, resulting in many new methods and applications. However, with this increase in published work and past surveys focusing mainly on methodology, a lack of standardisation became more prominent in the field. This non-standardisation is evident in the use of environments, which appear in no more than two works, and evaluation processes, such as qualitative analysis, that have become rare in current literature. In this survey, we systematically review current imitation learning literature and present our findings by (i) classifying imitation learning techniques, environments and metrics by introducing novel taxonomies; (ii) reflecting on main problems from the literature; and (iii) presenting challenges and future directions for researchers.
翻译:模仿学习是一种智能体通过模仿一个或多个教师的行为来学习执行任务的方法。该方法通过在依赖具有任务执行部分知识的教师进行学习,以及在必要时偏离其示例(例如处理教师样本中未出现的状态)之间取得平衡,从而在学习新任务所需时间与收集教师样本所需工作量之间实现了折中。近年来,模仿学习领域受到研究者广泛关注,产生了众多新方法与应用。然而,随着相关研究成果的持续增长,且以往综述主要聚焦于方法论,该领域缺乏标准化的问题日益凸显。这种非标准化现象体现在两个方面:一是环境的使用上,绝大多数环境仅出现在不超过两项研究中;二是评估流程上,诸如定性分析等方法在当前文献中已较为罕见。在本综述中,我们系统梳理了当前模仿学习文献,并通过以下方式呈现研究发现:(i)引入新的分类体系,对模仿学习技术、环境与度量进行归类;(ii)反思文献中的主要问题;(iii)为研究者指出挑战与未来方向。