Deep reinforcement learning (DRL), leveraging Deep Learning (DL) in reinforcement learning, has shown significant potential in achieving human-level autonomy in a wide range of domains, including robotics, computer vision, and computer games. This potential justifies the enthusiasm and growing interest in DRL in both academia and industry. However, the community currently focuses mostly on the development phase of DRL systems, with little attention devoted to DRL deployment. In this paper, we propose an empirical study on Stack Overflow (SO), the most popular Q&A forum for developers, to uncover and understand the challenges practitioners faced when deploying DRL systems. Specifically, we categorized relevant SO posts by deployment platforms: server/cloud, mobile/embedded system, browser, and game engine. After filtering and manual analysis, we examined 357 SO posts about DRL deployment, investigated the current state, and identified the challenges related to deploying DRL systems. Then, we investigate the prevalence and difficulty of these challenges. Results show that the general interest in DRL deployment is growing, confirming the study's relevance and importance. Results also show that DRL deployment is more difficult than other DRL issues. Additionally, we built a taxonomy of 31 unique challenges in deploying DRL to different platforms. On all platforms, RL environment-related challenges are the most popular, and communication-related challenges are the most difficult among practitioners. We hope our study inspires future research and helps the community overcome the most common and difficult challenges practitioners face when deploying DRL systems.

翻译：深度强化学习(DRL)将深度学习(DL)应用于强化学习，已在机器人技术、计算机视觉和电子游戏等多个领域展现出实现人类级自主性的巨大潜力。这种潜力激发了学术界和工业界对DRL的热情与日益增长的兴趣。然而，当前研究主要聚焦于DRL系统的开发阶段，对其部署环节的关注甚少。本文针对开发者最常用的问答论坛Stack Overflow(SO)开展实证研究，旨在揭示并理解从业者在部署DRL系统时面临的挑战。具体而言，我们按部署平台将相关SO帖子归类为：服务器/云端、移动/嵌入式系统、浏览器和游戏引擎。通过筛选与人工分析，我们研究了357篇关于DRL部署的SO帖子，梳理了当前现状，识别了与DRL系统部署相关的挑战，进而考察了这些挑战的普遍性与难度。结果表明，对DRL部署的整体兴趣正在增长，证实了本研究的现实重要性与相关性。结果还显示，DRL部署比其他DRL相关问题更为困难。此外，我们构建了针对不同平台部署DRL的31项独特挑战分类法。在所有平台中，与强化学习环境相关的挑战最为普遍，而通信相关挑战对从业者而言难度最大。本研究成果有望启发未来研究，并帮助该领域克服从业者在部署DRL系统时面临的最常见且最棘手的挑战。