Optimal control of parametric partial differential equations (PDEs) is crucial in many applications in engineering and science. In recent years, the progress in scientific machine learning has opened up new frontiers for the control of parametric PDEs. In particular, deep reinforcement learning (DRL) has the potential to solve high-dimensional and complex control problems in a large variety of applications. Most DRL methods rely on deep neural network (DNN) control policies. However, for many dynamical systems, DNN-based control policies tend to be over-parametrized, which means they need large amounts of training data, show limited robustness, and lack interpretability. In this work, we leverage dictionary learning and differentiable L$_0$ regularization to learn sparse, robust, and interpretable control policies for parametric PDEs. Our sparse policy architecture is agnostic to the DRL method and can be used in different policy-gradient and actor-critic DRL algorithms without changing their policy-optimization procedure. We test our approach on the challenging tasks of controlling parametric Kuramoto-Sivashinsky and convection-diffusion-reaction PDEs. We show that our method (1) outperforms baseline DNN-based DRL policies, (2) allows for the derivation of interpretable equations of the learned optimal control laws, and (3) generalizes to unseen parameters of the PDE without retraining the policies.

翻译：参数化偏微分方程的最优控制在工程与科学领域具有重要应用。近年来，科学机器学习的进展为参数化PDE控制开辟了新前沿。其中，深度强化学习在解决各类应用中的高维复杂控制问题方面展现出潜力。多数DRL方法依赖深度神经网络控制策略，然而对于诸多动力系统，基于DNN的控制策略往往存在参数冗余，导致其需要大量训练数据、鲁棒性有限且缺乏可解释性。本研究利用字典学习与可微L$_0$正则化方法，为参数化PDE学习稀疏、鲁棒且可解释的控制策略。所提出的稀疏策略架构与具体DRL方法无关，可在不改变策略优化流程的前提下，适配不同策略梯度与演员-评论家DRL算法。我们将该方法应用于参数化Kuramoto-Sivashinsky方程与对流-扩散-反应方程等挑战性控制任务，结果表明：(1)该方法优于基于DNN的基线DRL策略；(2)可推导所学最优控制律的可解释方程；(3)无需重新训练策略即可泛化至未见过的PDE参数。