Deep reinforcement learning (RL) algorithms are powerful tools for solving visuomotor decision tasks. However, the trained models are often difficult to interpret, because they are represented as end-to-end deep neural networks. In this paper, we shed light on the inner workings of such trained models by analyzing the pixels that they attend to during task execution, and comparing them with the pixels attended to by humans executing the same tasks. To this end, we investigate the following two questions that, to the best of our knowledge, have not been previously studied. 1) How similar are the visual representations learned by RL agents and humans when performing the same task? and, 2) How do similarities and differences in these learned representations explain RL agents' performance on these tasks? Specifically, we compare the saliency maps of RL agents against visual attention models of human experts when learning to play Atari games. Further, we analyze how hyperparameters of the deep RL algorithm affect the learned representations and saliency maps of the trained agents. The insights provided have the potential to inform novel algorithms for closing the performance gap between human experts and RL agents.

翻译：深度强化学习(RL)算法是解决相对自动决定任务的有力工具。然而,经过培训的模型往往很难解释,因为它们是端至端深神经网络。在本文中,我们通过分析这些经过培训的模型在任务执行期间所关注的像素,并把它们与执行同样任务的人类所关注的像素进行比较,来说明这些模型的内部作用。为此,我们调查了以下两个问题,根据我们的知识,我们以前没有研究过这两个问题。 1)在从事同样任务时,RL代理人和人类所学会的视觉表现如何相似? 和2)这些有学识的模型的相似性和差异如何解释RL代理人在这些任务上的绩效?具体地说,我们比较了RL代理人的显性图和人类专家在玩阿塔里游戏时所关注的视觉模型。此外,我们分析了深RL算法的超参数如何影响受过培训的代理人所了解的表象和突出的地图。所提供的洞察力有可能为缩小人类专家与RL代理人之间的性能差距提供的新算法。