Approaches for teaching learning agents via human demonstrations have been widely studied and successfully applied to multiple domains. However, the majority of imitation learning work utilizes only behavioral information from the demonstrator, i.e. which actions were taken, and ignores other useful information. In particular, eye gaze information can give valuable insight towards where the demonstrator is allocating visual attention, and holds the potential to improve agent performance and generalization. In this work, we propose Gaze Regularized Imitation Learning (GRIL), a novel context-aware, imitation learning architecture that learns concurrently from both human demonstrations and eye gaze to solve tasks where visual attention provides important context. We apply GRIL to a visual navigation task, in which an unmanned quadrotor is trained to search for and navigate to a target vehicle in a photorealistic simulated environment. We show that GRIL outperforms several state-of-the-art gaze-based imitation learning algorithms, simultaneously learns to predict human visual attention, and generalizes to scenarios not present in the training data. Supplemental videos and code can be found at https://sites.google.com/view/gaze-regularized-il/.

翻译：利用人类示范教授学习智能体的方法已被广泛研究并成功应用于多个领域。然而，大多数模仿学习工作仅利用示范者的行为信息（即执行的动作），而忽略了其他有用信息。特别是，眼动注视信息能提供关于示范者视觉注意力分配的重要线索，并具有提升智能体性能与泛化能力的潜力。本文提出一种新颖的上下文感知模仿学习架构——注视正则化模仿学习（GRIL），该架构同时从人类示范与眼动注视中学习，以解决视觉注意力提供关键背景信息的任务。我们将GRIL应用于视觉导航任务，在该任务中，无人四旋翼飞行器被训练在逼真的模拟环境中搜索并导航至目标车辆。实验表明，GRIL性能优于多种最先进的基于注视的模仿学习算法，能同步学习预测人类视觉注意力，并可泛化至训练数据中未出现的场景。补充视频与代码见 https://sites.google.com/view/gaze-regularized-il/。