Diffusion models surpass previous generative models in sample quality and training stability. Recent works have shown the advantages of diffusion models in improving reinforcement learning (RL) solutions. This survey aims to provide an overview of this emerging field and hopes to inspire new avenues of research. First, we examine several challenges encountered by RL algorithms. Then, we present a taxonomy of existing methods based on the roles of diffusion models in RL and explore how the preceding challenges are addressed. We further outline successful applications of diffusion models in various RL-related tasks. Finally, we conclude the survey and offer insights into future research directions. We are actively maintaining a GitHub repository for papers and other related resources in utilizing diffusion models in RL: https://github.com/apexrl/Diff4RLSurvey.

翻译：扩散模型在样本质量和训练稳定性方面超越了以往的生成模型。近期研究表明，扩散模型在改进强化学习解决方案方面具有显著优势。本综述旨在概述这一新兴领域，并期望激发新的研究方向。首先，我们探讨了强化学习算法面临的若干挑战。随后，基于扩散模型在强化学习中的作用，提出了一种现有方法的分类体系，并分析了这些方法如何应对前述挑战。我们进一步梳理了扩散模型在各类强化学习相关任务中的成功应用。最后，总结全文并对未来研究方向提出见解。我们持续维护一个关于扩散模型在强化学习中应用的论文及相关资源GitHub仓库：https://github.com/apexrl/Diff4RLSurvey。