We introduce controlgym, a library of thirty-six industrial control settings, and ten infinite-dimensional partial differential equation (PDE)-based control problems. Integrated within the OpenAI Gym/Gymnasium (Gym) framework, controlgym allows direct applications of standard reinforcement learning (RL) algorithms like stable-baselines3. Our control environments complement those in Gym with continuous, unbounded action and observation spaces, motivated by real-world control applications. Moreover, the PDE control environments uniquely allow the users to extend the state dimensionality of the system to infinity while preserving the intrinsic dynamics. This feature is crucial for evaluating the scalability of RL algorithms for control. This project serves the learning for dynamics & control (L4DC) community, aiming to explore key questions: the convergence of RL algorithms in learning control policies; the stability and robustness issues of learning-based controllers; and the scalability of RL algorithms to high- and potentially infinite-dimensional systems. We open-source the controlgym project at https://github.com/xiangyuan-zhang/controlgym.

翻译：我们介绍controlgym，一个包含三十六个工业控制场景和十个基于无穷维偏微分方程（PDE）的控制问题的库。controlgym集成在OpenAI Gym/Gymnasium（Gym）框架内，允许直接应用诸如stable-baselines3等标准强化学习（RL）算法。我们的控制环境补充了Gym中具有连续、无界动作和观测空间的环境，这些环境源于实际控制应用。此外，PDE控制环境独特地允许用户将系统的状态维度扩展到无穷大，同时保持内在动力学特性。该特性对于评估RL算法在控制中的可扩展性至关重要。本项目服务于学习与动力学控制（L4DC）社区，旨在探索关键问题：RL算法在学习控制策略中的收敛性；基于学习的控制器的稳定性与鲁棒性问题；以及RL算法对高维乃至无穷维系统的可扩展性。我们在https://github.com/xiangyuan-zhang/controlgym上开源了controlgym项目。