Model-based approaches are becoming increasingly popular in the field of offline reinforcement learning, with high potential in real-world applications due to the model's capability of thoroughly utilizing the large historical datasets available with supervised learning techniques. This paper presents a literature review of recent work in offline model-based reinforcement learning, a field that utilizes model-based approaches in offline reinforcement learning. The survey provides a brief overview of the concepts and recent developments in both offline reinforcement learning and model-based reinforcement learning, and discuss the intersection of the two fields. We then presents key relevant papers in the field of offline model-based reinforcement learning and discuss their methods, particularly their approaches in solving the issue of distributional shift, the main problem faced by all current offline model-based reinforcement learning methods. We further discuss key challenges faced by the field, and suggest possible directions for future work.

翻译：基于模型的方法在离线强化学习领域日益流行，由于模型能够充分利用监督学习技术处理大规模历史数据集，在现实应用中具有巨大潜力。本文对离线模型驱动强化学习领域的最新研究成果进行了文献综述，该领域致力于将基于模型的方法应用于离线强化学习。本综述简要概述了离线强化学习与基于模型强化学习的概念及最新进展，并探讨了两者的交叉领域。随后，我们系统梳理了离线模型驱动强化学习领域的关键论文，重点分析其方法，特别是应对分布偏移问题的策略——这是当前所有离线模型驱动强化学习方法面临的核心挑战。最后，我们进一步探讨了该领域面临的主要难点，并提出了未来可能的研究方向。