Machine Learning (ML) is increasingly being adopted in different industries. Deep Reinforcement Learning (DRL) is a subdomain of ML used to produce intelligent agents. Despite recent developments in DRL technology, the main challenges that developers face in the development of DRL applications are still unknown. To fill this gap, in this paper, we conduct a large-scale empirical study of 927 DRL-related posts extracted from Stack Overflow, the most popular Q&A platform in the software community. Through the process of labeling and categorizing extracted posts, we created a taxonomy of common challenges encountered in the development of DRL applications, along with their corresponding popularity levels. This taxonomy has been validated through a survey involving 59 DRL developers. Results show that at least 45% of developers experienced 18 of the 21 challenges identified in the taxonomy. The most frequent source of difficulty during the development of DRL applications are Comprehension, API usage, and Design problems, while Parallel processing, and DRL libraries/frameworks are classified as the most difficult challenges to address, with respect to the time required to receive an accepted answer. We hope that the research community will leverage this taxonomy to develop efficient strategies to address the identified challenges and improve the quality of DRL applications.

翻译：机器学习（ML）正日益被各行业所采纳。深度强化学习（DRL）作为ML的一个子领域，用于生成智能体。尽管DRL技术近年来有所发展，开发者在DRL应用开发过程中面临的主要挑战尚不明确。为填补这一空白，本文对从软件社区最流行的问答平台Stack Overflow中提取的927篇DRL相关帖子进行了大规模实证研究。通过对所提取帖子进行标注与分类，我们构建了DRL应用开发中常见挑战的分类体系，并附带了各挑战对应的流行程度。该分类体系已通过一项涉及59名DRL开发者的调查得到验证。结果表明，至少45%的开发者经历过分类体系所识别的21项挑战中的18项。DRL应用开发中最频繁的困难来源是理解、API使用和设计问题，而并行处理以及DRL库/框架则被归类为最难以应对的挑战（基于获得被采纳答案所需时间）。我们期望研究社区能利用这一分类体系，制定有效策略以应对已识别的挑战，从而提升DRL应用的质量。