Imitation learning (IL) enables agents to mimic expert behaviors. Most previous IL techniques focus on precisely imitating one policy through mass demonstrations. However, in many applications, what humans require is the ability to perform various tasks directly through a few demonstrations of corresponding tasks, where the agent would meet many unexpected changes when deployed. In this scenario, the agent is expected to not only imitate the demonstration but also adapt to unforeseen environmental changes. This motivates us to propose a new topic called imitator learning (ItorL), which aims to derive an imitator module that can on-the-fly reconstruct the imitation policies based on very limited expert demonstrations for different unseen tasks, without any extra adjustment. In this work, we focus on imitator learning based on only one expert demonstration. To solve ItorL, we propose Demo-Attention Actor-Critic (DAAC), which integrates IL into a reinforcement-learning paradigm that can regularize policies' behaviors in unexpected situations. Besides, for autonomous imitation policy building, we design a demonstration-based attention architecture for imitator policy that can effectively output imitated actions by adaptively tracing the suitable states in demonstrations. We develop a new navigation benchmark and a robot environment for \topic~and show that DAAC~outperforms previous imitation methods \textit{with large margins} both on seen and unseen tasks.

翻译：摘要：模仿学习（IL）能使智能体模仿专家行为。先前的大多数IL技术专注于通过大量示范精确模仿单一策略。然而在许多应用中，人类需要的是通过对应任务的少量示范直接执行各种任务的能力，而智能体在部署时会遭遇许多意外变化。在此场景下，智能体不仅需模仿示范行为，还需适应不可预知的环境变化。这促使我们提出名为模仿器学习（ItorL）的新课题，旨在构建一个模仿器模块，该模块能根据极少量的专家示范，针对不同的未见任务即时重构模仿策略，无需任何额外调整。本文聚焦于仅基于一个专家示范的模仿器学习。为解决ItorL，我们提出演示注意动作评价（DAAC）算法，将IL整合到强化学习范式之中，从而能在意外情境中规范策略行为。此外，为构建自主模仿策略，我们设计了一种基于示范注意力的架构，该架构能通过自适应追踪示范中的合适状态，有效输出模仿动作。我们为这一课题开发了新的导航基准测试环境和机器人仿真环境，并证明DAAC在已见和未见任务上均以显著优势超越先前模仿方法。