(Economic) nonlinear model predictive control ((e)NMPC) requires dynamic models that are sufficiently accurate and computationally tractable. Data-driven surrogate models for mechanistic models can reduce the computational burden of (e)NMPC; however, such models are typically trained by system identification for maximum prediction accuracy on simulation samples and perform suboptimally in (e)NMPC. We present a method for end-to-end reinforcement learning of Koopman surrogate models for optimal performance as part of (e)NMPC. We apply our method to two applications derived from an established nonlinear continuous stirred-tank reactor model. The controller performance is compared to that of (e)NMPCs utilizing models trained using system identification, and model-free neural network controllers trained using reinforcement learning. We show that the end-to-end trained models outperform those trained using system identification in (e)NMPC, and that, in contrast to the neural network controllers, the (e)NMPC controllers can react to changes in the control setting without retraining.

翻译：（经济）非线性模型预测控制（(e)NMPC）需要兼具高精度与计算可行性的动态模型。基于数据驱动的代理模型替代机理模型可降低(e)NMPC的计算负担，然而此类模型通常通过系统辨识方法训练以最大化仿真样本的预测精度，导致其在(e)NMPC中性能次优。本文提出一种端到端强化学习方法，通过优化Koopman代理模型实现(e)NMPC的最优性能。我们将该方法应用于基于经典非线性连续搅拌反应釜模型的两个实际场景。控制器性能与两类基准进行对比：采用系统辨识训练模型的(e)NMPC控制器，以及采用强化学习训练的无模型神经网络控制器。实验表明：在(e)NMPC框架下，端到端训练的模型性能优于系统辨识训练的模型；与神经网络控制器不同，(e)NMPC控制器无需重新训练即可响应控制设置的动态变化。