With the advent of universal function approximators in the domain of reinforcement learning, the number of practical applications leveraging deep reinforcement learning (DRL) has exploded. Decision-making in automated driving tasks has emerged as a chief application among them, taking the sensor data or the higher-order kinematic variables as the input and providing a discrete choice or continuous control output. However, the black-box nature of the models presents an overwhelming limitation that restricts the real-world deployment of DRL in autonomous vehicles (AVs). Therefore, in this research work, we focus on the interpretability of an attention-based DRL framework. We use a continuous proximal policy optimization-based DRL algorithm as the baseline model and add a multi-head attention framework in an open-source AV simulation environment. We provide some analytical techniques for discussing the interpretability of the trained models in terms of explainability and causality for spatial and temporal correlations. We show that the weights in the first head encode the positions of the neighboring vehicles while the second head focuses on the leader vehicle exclusively. Also, the ego vehicle's action is causally dependent on the vehicles in the target lane spatially and temporally. Through these findings, we reliably show that these techniques can help practitioners decipher the results of the DRL algorithms.

翻译：随着通用函数逼近器在强化学习领域的出现，基于深度强化学习的实际应用数量激增。在自动化驾驶任务中，决策制定已成为主要应用之一，其接收传感器数据或高阶运动学变量作为输入，并提供离散选择或连续控制输出。然而，模型的黑箱特性构成了重大限制，阻碍了深度强化学习在自动驾驶车辆中的实际部署。因此，在本研究中，我们聚焦于基于注意力机制的深度强化学习框架的可解释性。我们采用基于连续近端策略优化的深度强化学习算法作为基线模型，并在开源自动驾驶模拟环境中引入多头注意力框架。我们提供了一些分析技术，从时空相关性的可解释性和因果性角度探讨训练模型的可理解性。结果表明，第一个注意力头的权重编码了相邻车辆的位置，而第二个注意力头则专门关注前车。此外，自车的行动在空间和时间上因果依赖于目标车道上的车辆。通过这些发现，我们可靠地证明这些技术可帮助实践者解读深度强化学习算法的结果。