Machine Learning (ML) is increasingly being adopted in different industries. Deep Reinforcement Learning (DRL) is a subdomain of ML used to produce intelligent agents. Despite recent developments in DRL technology, the main challenges that developers face in the development of DRL applications are still unknown. To fill this gap, in this paper, we conduct a large-scale empirical study of 927 DRL-related posts extracted from Stack Overflow, the most popular Q&A platform in the software community. Through the process of labeling and categorizing extracted posts, we created a taxonomy of common challenges encountered in the development of DRL applications, along with their corresponding popularity levels. This taxonomy has been validated through a survey involving 65 DRL developers. Results show that at least 45% of developers experienced 18 of the 21 challenges identified in the taxonomy. The most frequent source of difficulty during the development of DRL applications are Comprehension, API usage, and Design problems, while Parallel processing, and DRL libraries/frameworks are classified as the most difficult challenges to address, with respect to the time required to receive an accepted answer. We hope that the research community will leverage this taxonomy to develop efficient strategies to address the identified challenges and improve the quality of DRL applications.

翻译：机器学习正日益被各行业广泛采用。深度强化学习作为机器学习的一个子领域，用于生成智能体。尽管深度强化学习技术近年有所发展，但开发者在开发深度强化学习应用时面临的主要挑战仍不明确。为填补这一空白，本文对软件社区最流行的问答平台Stack Overflow上提取的927个与深度强化学习相关的帖子进行了大规模实证研究。通过对提取的帖子进行标记和分类，我们构建了深度强化学习应用开发中常见挑战的分类体系，并标注了各挑战的流行程度。该分类体系通过一项涉及65位深度强化学习开发者的调查验证。结果显示，在分类体系确定的21个挑战中，至少45%的开发者经历过其中18个。深度强化学习应用开发中最常见的困难来源是理解、API使用和设计问题，而并行处理以及深度强化学习库/框架则是根据获得被采纳答案所需时间被归类为最难解决的挑战。我们希望研究社区能利用此分类体系开发有效策略来应对已识别的挑战，并提升深度强化学习应用的质量。