Over the last decade, data-driven methods have surged in popularity, emerging as valuable tools for control theory. As such, neural network approximations of control feedback laws, system dynamics, and even Lyapunov functions have attracted growing attention. With the ascent of learning based control, the need for accurate, fast, and easy-to-use benchmarks has increased. In this work, we present the first learning-based environment for boundary control of PDEs. In our benchmark, we introduce three foundational PDE problems - a 1D transport PDE, a 1D reaction-diffusion PDE, and a 2D Navier-Stokes PDE - whose solvers are bundled in an user-friendly reinforcement learning gym. With this gym, we then present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems, achieving stability, although at a higher cost compared to model-based PDE backstepping. With the set of benchmark environments and detailed examples, this work significantly lowers the barrier to entry for learning-based PDE control - a topic largely unexplored by the data-driven control community. The entire benchmark is available on Github along with detailed documentation and the presented reinforcement learning models are open sourced.

翻译：过去十年间，数据驱动方法迅速普及，成为控制理论的重要工具。基于神经网络的反馈控制律、系统动力学乃至李雅普诺夫函数近似方法日益受到关注。随着基于学习的控制方法兴起，对精准、快速且易用的基准测试需求与日俱增。本文首次构建了基于学习环境的偏微分方程边界控制测试平台。该基准测试引入三类基础PDE问题——一维输运PDE、一维反应扩散PDE及二维纳维-斯托克斯PDE，其求解器集成于用户友好的强化学习健身房中。利用此平台，我们首次提出解决该系列基准问题的无模型强化学习算法，虽较基于模型的PDE反步法成本更高，但实现了稳定性控制。通过构建基准环境集与详细案例，本研究显著降低了学习型PDE控制的入门门槛——这一数据驱动控制领域鲜有探索的方向。整套基准测试代码已在Github开放，附带完整文档，所提出的强化学习模型均以开源形式发布。