Deep reinforcement learning has led to dramatic breakthroughs in the field of artificial intelligence for the past few years. As the amount of rollout experience data and the size of neural networks for deep reinforcement learning have grown continuously, handling the training process and reducing the time consumption using parallel and distributed computing is becoming an urgent and essential desire. In this paper, we perform a broad and thorough investigation on training acceleration methodologies for deep reinforcement learning based on parallel and distributed computing, providing a comprehensive survey in this field with state-of-the-art methods and pointers to core references. In particular, a taxonomy of literature is provided, along with a discussion of emerging topics and open issues. This incorporates learning system architectures, simulation parallelism, computing parallelism, distributed synchronization mechanisms, and deep evolutionary reinforcement learning. Further, we compare 16 current open-source libraries and platforms with criteria of facilitating rapid development. Finally, we extrapolate future directions that deserve further research.

翻译：过去几年中，深度强化学习在人工智能领域取得了突破性进展。随着深度强化学习中经验采样数据量的持续增长以及神经网络规模的不断扩大，利用并行与分布式计算技术处理训练过程并降低时间消耗已成为迫切且关键的需求。本文对基于并行与分布式计算的深度强化学习训练加速方法进行了广泛而深入的调研，通过梳理该领域的前沿方法与核心文献，提供了系统性的综述。具体而言，本文建立了文献分类体系，并探讨了新兴议题与开放性问题，涵盖学习系统架构、仿真并行化、计算并行化、分布式同步机制以及深度进化强化学习等方面。此外，我们依据促进快速开发的标准，对当前16个开源库与平台进行了比较分析。最后，本文对未来值得深入研究的方向进行了展望。