Reinforcement learning (RL) agents need to explore their environments in order to learn optimal policies. In many environments and tasks, safety is of critical importance. The widespread use of simulators offers a number of advantages, including safe exploration which will be inevitable in cases when RL systems need to be trained directly in the physical environment (e.g. in human-robot interaction). The popular Safety Gym library offers three mobile agent types that can learn goal-directed tasks while considering various safety constraints. In this paper, we extend the applicability of safe RL algorithms by creating a customized environment with Panda robotic arm where Safety Gym algorithms can be tested. We performed pilot experiments with the popular PPO algorithm comparing the baseline with the constrained version and show that the constrained version is able to learn the equally good policy while better complying with safety constraints and taking longer training time as expected.

翻译：强化学习（RL）智能体需要探索环境以学习最优策略。在许多环境和任务中，安全性至关重要。模拟器的广泛使用提供了多项优势，包括安全探索，这在RL系统需直接在物理环境（如人机交互）中训练时将成为必然需求。流行的Safety Gym库提供了三种可移动智能体类型，能在考虑各种安全约束的同时学习目标导向任务。本文通过创建包含Panda机器人臂的定制化环境来扩展安全RL算法的适用性，该环境可用于测试Safety Gym算法。我们使用流行的PPO算法进行了初步实验，将基线版本与约束版本进行对比，结果表明约束版本能够学习到同等优秀的策略，同时更好地遵守安全约束，且训练时间如预期般更长。