Industry 4.0 is driven by demands like shorter time-to-market, mass customization of products, and batch size one production. Reinforcement Learning (RL), a machine learning paradigm shown to possess a great potential in improving and surpassing human level performance in numerous complex tasks, allows coping with the mentioned demands. In this paper, we present an OPC UA based Operational Technology (OT)-aware RL architecture, which extends the standard RL setting, combining it with the setting of digital twins. Moreover, we define an OPC UA information model allowing for a generalized plug-and-play like approach for exchanging the RL agent used. In conclusion, we demonstrate and evaluate the architecture, by creating a proof of concept. By means of solving a toy example, we show that this architecture can be used to determine the optimal policy using a real control system.

翻译：工业4.0的驱动力包括缩短产品上市时间、产品大规模定制以及单批次生产等需求。强化学习作为一种机器学习范式，在众多复杂任务中展现出提升甚至超越人类水平的巨大潜力，能够应对上述需求。本文提出了一种基于OPC UA的运营技术感知型强化学习架构，该架构扩展了标准强化学习设置，并将其与数字孪生环境相结合。此外，我们定义了一个OPC UA信息模型，支持通过通用的即插即用方式交换所使用的强化学习智能体。最后，我们通过概念验证演示并评估了该架构。通过解决一个示例性问题，我们证明该架构能够利用实际控制系统确定最优策略。