The energy management problem in the context of smart grids is inherently complex due to the interdependencies among diverse system components. Although Reinforcement Learning (RL) has been proposed for solving Optimal Power Flow (OPF) problems, the requirement for iterative interaction with an environment often necessitates computationally expensive simulators, leading to significant sample inefficiency. In this study, these challenges are addressed through the use of Physics-Informed Neural Networks (PINNs), which can replace conventional and costly smart grid simulators. The RL policy learning process is enhanced so that convergence can be achieved in a fraction of the time required by the original environment. The PINN-based surrogate is compared with other benchmark data-driven surrogate models. By incorporating knowledge of the underlying physical laws, the results show that the PINN surrogate is the only approach considered in this context that can obtain a strong RL policy even without access to samples from the true simulator. The results demonstrate that using PINN surrogates can accelerate training by 50% compared to RL training without a surrogate. This approach enables the rapid generation of performance scores similar to those produced by the original simulator.

翻译：智能电网背景下的能源管理问题因其内部各系统组件间的相互依赖关系而具有固有的复杂性。尽管强化学习已被提出用于求解最优潮流问题，但其与环境的迭代交互需求通常需要计算成本高昂的仿真器，导致显著的样本效率低下。本研究通过采用物理信息神经网络来解决这些挑战，该网络能够替代传统且昂贵的智能电网仿真器。强化学习的策略学习过程得到增强，使得收敛所需时间仅为原始环境所需时间的一小部分。研究将基于物理信息神经网络的代理模型与其他基准数据驱动代理模型进行了比较。通过融入底层物理规律的知识，结果表明在此背景下，物理信息神经网络代理是唯一能够在无需真实仿真器样本的情况下仍能获得强有力强化学习策略的方法。实验证明，与不使用代理模型的强化学习训练相比，采用物理信息神经网络代理可使训练速度提升50%。该方法能够快速生成与原始仿真器产出相近的性能评分。