Due to the trial-and-error nature, it is typically challenging to apply RL algorithms to safety-critical real-world applications, such as autonomous driving, human-robot interaction, robot manipulation, etc, where such errors are not tolerable. Recently, safe RL (i.e. constrained RL) has emerged rapidly in the literature, in which the agents explore the environment while satisfying constraints. Due to the diversity of algorithms and tasks, it remains difficult to compare existing safe RL algorithms. To fill that gap, we introduce GUARD, a Generalized Unified SAfe Reinforcement Learning Development Benchmark. GUARD has several advantages compared to existing benchmarks. First, GUARD is a generalized benchmark with a wide variety of RL agents, tasks, and safety constraint specifications. Second, GUARD comprehensively covers state-of-the-art safe RL algorithms with self-contained implementations. Third, GUARD is highly customizable in tasks and algorithms. We present a comparison of state-of-the-art safe RL algorithms in various task settings using GUARD and establish baselines that future work can build on.

翻译：受试错本质的影响，强化学习算法通常难以应用于安全关键的现实场景（如自动驾驶、人机交互、机器人操作等），这类场景不容许算法出错。近年来，安全强化学习（即约束强化学习）在文献中迅速兴起，其中智能体在满足约束条件的同时探索环境。由于算法和任务的多样性，现有安全强化学习算法的比较仍存在困难。为填补这一空白，我们提出GUARD——一个通用统一安全强化学习开发基准。相较现有基准，GUARD具备以下优势：首先，GUARD是一个通用基准，涵盖多种强化学习智能体、任务及安全约束规范；其次，GUARD全面收录了最先进的安全强化学习算法并附带独立实现；再次，GUARD在任务和算法层面均具有高度可定制性。我们利用GUARD展示了多种任务设定下最先进安全强化学习算法的对比结果，并建立了可供未来研究参考的基线。