Control of distributed parameter systems affected by delays is a challenging task, particularly when the delays depend on spatial variables. The idea of integrating analytical control theory with learning-based control within a unified control scheme is becoming increasingly promising and advantageous. In this paper, we address the problem of controlling an unstable first-order hyperbolic PDE with spatially-varying delays by combining PDE backstepping control strategies and deep reinforcement learning (RL). To eliminate the assumption on the delay function required for the backstepping design, we propose a soft actor-critic (SAC) architecture incorporating a DeepONet to approximate the backstepping controller. The DeepONet extracts features from the backstepping controller and feeds them into the policy network. In simulations, our algorithm outperforms the baseline SAC without prior backstepping knowledge and the analytical controller.

翻译：控制受延迟影响的分布参数系统是一项具有挑战性的任务，特别是当延迟依赖于空间变量时。将解析控制理论与基于学习的控制方法整合到一个统一控制框架中的思路正变得越来越有前景和优势。本文通过结合偏微分方程反步控制策略与深度强化学习，解决了控制具有空间变化延迟的不稳定一阶双曲型偏微分方程的问题。为了消除反步设计中对延迟函数所做的假设，我们提出了一种结合DeepONet的软演员-评论家架构，用以逼近反步控制器。该DeepONet从反步控制器中提取特征，并将其馈送到策略网络中。仿真结果表明，我们的算法在性能上优于不具备先验反步知识的基线SAC以及解析控制器。