Due to the trial-and-error nature, it is typically challenging to apply RL algorithms to safety-critical real-world applications, such as autonomous driving, human-robot interaction, robot manipulation, etc, where such errors are not tolerable. Recently, safe RL (i.e. constrained RL) has emerged rapidly in the literature, in which the agents explore the environment while satisfying constraints. Due to the diversity of algorithms and tasks, it remains difficult to compare existing safe RL algorithms. To fill that gap, we introduce GUARD, a Generalized Unified SAfe Reinforcement Learning Development Benchmark. GUARD has several advantages compared to existing benchmarks. First, GUARD is a generalized benchmark with a wide variety of RL agents, tasks, and safety constraint specifications. Second, GUARD comprehensively covers state-of-the-art safe RL algorithms with self-contained implementations. Third, GUARD is highly customizable in tasks and algorithms. We present a comparison of state-of-the-art safe RL algorithms in various task settings using GUARD and establish baselines that future work can build on.

翻译：由于探索与试错的天性，强化学习算法通常难以直接应用于自动驾驶、人机交互、机器人操作等安全关键的真实世界场景，因为此类错误是不可容忍的。近年来，安全强化学习（即约束强化学习）在文献中迅速发展，其中智能体在满足约束条件的同时探索环境。由于算法和任务的多样性，现有安全强化学习算法之间的比较仍然困难。为填补这一空白，我们提出GUARD，一个通用统一的强化学习开发基准。相较于现有基准，GUARD具有以下优势：首先，GUARD是一个通用基准，涵盖多种强化学习智能体、任务和安全约束规范。其次，GUARD全面收录了最先进的安全强化学习算法，并提供完整的实现代码。第三，GUARD在任务和算法上具有高度可定制性。我们使用GUARD在多种任务设置下比较了最先进的安全强化学习算法，并建立了可供未来研究参考的基线。