Reinforcement Learning (RL) has shown remarkable success in real-world applications, particularly in robotics control. However, RL adoption remains limited due to insufficient safety guarantees. We introduce Nightmare Dreamer, a model-based Safe RL algorithm that addresses safety concerns by leveraging a learned world model to predict potential safety violations and plan actions accordingly. Nightmare Dreamer achieves nearly zero safety violations while maximizing rewards. Nightmare Dreamer outperforms model-free baselines on Safety Gymnasium tasks using only image observations, achieving nearly a 20x improvement in efficiency.

翻译：强化学习（RL）在现实世界应用中展现出卓越成效，尤其在机器人控制领域。然而，由于安全保证不足，强化学习的应用推广仍受限制。本文提出噩梦编织者——一种基于模型的安全强化学习算法，该算法通过利用学习得到的世界模型预测潜在安全违规行为并据此规划动作，从而应对安全性问题。噩梦编织者在最大化奖励的同时实现了近乎零安全违规。在仅使用图像观测的Safety Gymnasium任务中，噩梦编织者优于无模型基线方法，实现了近20倍的效率提升。