Over the last decade, data-driven methods have surged in popularity, emerging as valuable tools for control theory. As such, neural network approximations of control feedback laws, system dynamics, and even Lyapunov functions have attracted growing attention. With the ascent of learning based control, the need for accurate, fast, and easy-to-use benchmarks has increased. In this work, we present the first learning-based environment for boundary control of PDEs. In our benchmark, we introduce three foundational PDE problems - a 1D transport PDE, a 1D reaction-diffusion PDE, and a 2D Navier-Stokes PDE - whose solvers are bundled in an user-friendly reinforcement learning gym. With this gym, we then present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems, achieving stability, although at a higher cost compared to model-based PDE backstepping. With the set of benchmark environments and detailed examples, this work significantly lowers the barrier to entry for learning-based PDE control - a topic largely unexplored by the data-driven control community. The entire benchmark is available on Github along with detailed documentation and the presented reinforcement learning models are open sourced.

翻译：过去十年间，数据驱动方法迅速普及，成为控制理论领域的重要工具。因此，控制反馈律、系统动力学乃至李雅普诺夫函数的神经网络近似方法日益受到关注。随着学习型控制方法的兴起，对精确、快速且易于使用的基准测试的需求也日益增长。本研究提出了首个基于学习的偏微分方程边界控制环境。在该基准平台中，我们引入了三个基础偏微分方程问题——一维输运偏微分方程、一维反应扩散偏微分方程和二维纳维-斯托克斯偏微分方程——其求解器集成于用户友好的强化学习训练平台中。基于该平台，我们首次提出了一套无模型强化学习算法用于求解这一系列基准问题，虽然相比基于模型的偏微分方程反步法成本更高，但成功实现了系统稳定性。通过提供基准环境集与详细案例，本研究显著降低了学习型偏微分方程控制的研究门槛——这一主题在数据驱动控制领域尚未得到充分探索。完整基准测试已在GitHub平台开源，附有详细文档说明，所提出的强化学习模型均已开放源代码。