The use of Deep Reinforcement Learning (DRL) schemes has increased dramatically since their first introduction in 2015. Though uses in many different applications are being found they still have a problem with the lack of interpretability. This has bread a lack of understanding and trust in the use of DRL solutions from researchers and the general public. To solve this problem the field of explainable artificial intelligence (XAI) has emerged. This is a variety of different methods that look to open the DRL black boxes, they range from the use of interpretable symbolic decision trees to numerical methods like Shapley Values. This review looks at which methods are being used and what applications they are being used. This is done to identify which models are the best suited to each application or if a method is being underutilised.

翻译：自2015年首次引入以来,深强化学习(DRL)计划的使用急剧增加,尽管在很多不同的应用中发现其用途仍然缺乏可解释性。这使得研究人员和公众对DRL解决方案的使用缺乏理解和信任。为了解决这一问题,出现了可解释的人工智能领域。这是寻求打开DRL黑盒的各种不同方法,从可解释的象征性决定树到Shapley值等数字方法。本次审查考察了正在使用哪些方法以及它们正在使用什么应用。这样做是为了确定哪些模型最适合每种应用,或者如果一种方法使用不足。