We introduce controlgym, a library of thirty-six safety-critical industrial control settings, and ten infinite-dimensional partial differential equation (PDE)-based control problems. Integrated within the OpenAI Gym/Gymnasium (Gym) framework, controlgym allows direct applications of standard reinforcement learning (RL) algorithms like stable-baselines3. Our control environments complement those in Gym with continuous, unbounded action and observation spaces, motivated by real-world control applications. Moreover, the PDE control environments uniquely allow the users to extend the state dimensionality of the system to infinity while preserving the intrinsic dynamics. This feature is crucial for evaluating the scalability of RL algorithms for control. This project serves the learning for dynamics & control (L4DC) community, aiming to explore key questions: the convergence of RL algorithms in learning control policies; the stability and robustness issues of learning-based controllers; and the scalability of RL algorithms to high- and potentially infinite-dimensional systems. We open-source the controlgym project at https://github.com/xiangyuan-zhang/controlgym.

翻译：我们推出了controlgym，这是一个包含36个安全关键型工业控制场景及10个基于无穷维偏微分方程（PDE）控制问题的库。controlgym集成于OpenAI Gym/Gymnasium（Gym）框架内，可直接应用如stable-baselines3等标准强化学习（RL）算法。受实际控制应用启发，我们的控制环境通过连续、无界的动作与观测空间，对Gym中的现有环境形成补充。此外，PDE控制环境独具特色，允许用户在保留系统内在动力学特性的前提下，将状态维度扩展至无穷大。这一特性对于评估RL算法在控制问题中的可扩展性至关重要。本项目的目标服务于学习驱动动力学与控制（L4DC）社区，旨在探索关键问题：RL算法在学习控制策略时的收敛性；基于学习的控制器的稳定性与鲁棒性问题；以及RL算法对高维乃至无穷维系统的可扩展性。我们已在https://github.com/xiangyuan-zhang/controlgym上开源controlgym项目。