Explanation is a key component for the adoption of reinforcement learning (RL) in many real-world decision-making problems. In the literature, the explanation is often provided by saliency attribution to the features of the RL agent's state. In this work, we propose a complementary approach to these explanations, particularly for offline RL, where we attribute the policy decisions of a trained RL agent to the trajectories encountered by it during training. To do so, we encode trajectories in offline training data individually as well as collectively (encoding a set of trajectories). We then attribute policy decisions to a set of trajectories in this encoded space by estimating the sensitivity of the decision with respect to that set. Further, we demonstrate the effectiveness of the proposed approach in terms of quality of attributions as well as practical scalability in diverse environments that involve both discrete and continuous state and action spaces such as grid-worlds, video games (Atari) and continuous control (MuJoCo). We also conduct a human study on a simple navigation task to observe how their understanding of the task compares with data attributed for a trained RL policy. Keywords -- Explainable AI, Verifiability of AI Decisions, Explainable RL.

翻译：解释是强化学习在诸多现实决策问题中得以应用的关键组成部分。文献中，通常通过将显著性归因于强化学习智能体状态的特征来提供解释。本文提出了一种对这些解释的补充方法，尤其适用于离线强化学习场景，该方法将训练好的强化学习智能体的策略决策归因于其在训练过程中遇到的轨迹。为此，我们分别对离线训练数据中的单条轨迹以及多条轨迹（即轨迹集合）进行编码。然后，在该编码空间中，通过估计策略决策相对于轨迹集合的敏感性，将其归因于特定的轨迹集合。此外，我们通过归因质量及实际可扩展性两方面展示了所提方法的有效性，应用环境涵盖具有离散与连续状态及动作空间的多样化场景，例如网格世界、视频游戏（Atari）以及连续控制任务（MuJoCo）。我们还针对一项简单的导航任务进行了人类研究，以观察他们对任务的理解如何与为已训练强化学习策略归因的数据相比较。关键词——可解释人工智能、人工智能决策可验证性、可解释强化学习。