The use of Deep Reinforcement Learning (DRL) schemes has increased dramatically since their first introduction in 2015. Though uses in many different applications are being found, they still have a problem with the lack of interpretability. This has bread a lack of understanding and trust in the use of DRL solutions from researchers and the general public. To solve this problem, the field of Explainable Artificial Intelligence (XAI) has emerged. This entails a variety of different methods that look to open the DRL black boxes, ranging from the use of interpretable symbolic Decision Trees (DT) to numerical methods like Shapley Values. This review looks at which methods are being used and for which applications. This is done to identify which models are the best suited to each application or if a method is being underutilised.

翻译：深度强化学习（DRL）方案自2015年首次提出以来，其应用范围迅速扩大。尽管已在众多不同领域得到应用，但DRL仍存在缺乏可解释性的问题。这种状况导致研究人员和公众对DRL解决方案缺乏理解与信任。为解决此问题，可解释人工智能（XAI）领域应运而生。该领域包含多种旨在打开DRL黑箱的方法，从使用可解释的符号化决策树（DT）到诸如沙普利值（Shapley Values）等数值方法。本综述审视了当前正在使用的方法及其应用领域，旨在识别哪种模型最适合特定应用场景，或是否存在未被充分利用的方法。