Effectively controlling systems governed by Partial Differential Equations (PDEs) is crucial in several fields of Applied Sciences and Engineering. These systems usually yield significant challenges to conventional control schemes due to their nonlinear dynamics, partial observability, high-dimensionality once discretized, distributed nature, and the requirement for low-latency feedback control. Reinforcement Learning (RL), particularly Deep RL (DRL), has recently emerged as a promising control paradigm for such systems, demonstrating exceptional capabilities in managing high-dimensional, nonlinear dynamics. However, DRL faces challenges including sample inefficiency, robustness issues, and an overall lack of interpretability. To address these issues, we propose a data-efficient, interpretable, and scalable Dyna-style Model-Based RL framework for PDE control, combining the Sparse Identification of Nonlinear Dynamics with Control (SINDy-C) algorithm and an autoencoder (AE) framework for the sake of dimensionality reduction of PDE states and actions. This novel approach enables fast rollouts, reducing the need for extensive environment interactions, and provides an interpretable latent space representation of the PDE forward dynamics. We validate our method on two PDE problems describing fluid flows - namely, the 1D Burgers equation and 2D Navier-Stokes equations - comparing it against a model-free baseline, and carrying out an extensive analysis of the learned dynamics.

翻译：有效控制偏微分方程（PDE）所描述的系统在应用科学与工程的多个领域至关重要。这些系统通常因其非线性动力学、部分可观测性、离散化后的高维特性、分布式本质以及对低延迟反馈控制的要求，给传统控制方案带来重大挑战。强化学习（RL），特别是深度强化学习（DRL），近年来已成为此类系统一种有前景的控制范式，在管理高维非线性动力学方面展现出卓越能力。然而，DRL面临着样本效率低、鲁棒性问题以及整体可解释性不足等挑战。为解决这些问题，我们提出了一种数据高效、可解释且可扩展的基于模型的 Dyna 风格强化学习框架，用于 PDE 控制。该框架结合了带控制的非线性动力学稀疏辨识（SINDy-C）算法与自编码器（AE）框架，以实现 PDE 状态与动作的降维。这种新颖方法能够实现快速推演，减少对大量环境交互的需求，并提供 PDE 前向动力学的一个可解释的潜在空间表示。我们在两个描述流体流动的 PDE 问题（即一维 Burgers 方程和二维 Navier-Stokes 方程）上验证了我们的方法，将其与无模型基线进行比较，并对学习到的动力学进行了深入分析。