Recent challenges in operating power networks arise from increasing energy demands and unpredictable renewable sources like wind and solar. While reinforcement learning (RL) shows promise in managing these networks, through topological actions like bus and line switching, efficiently handling large action spaces as networks grow is crucial. This paper presents a hierarchical multi-agent reinforcement learning (MARL) framework tailored for these expansive action spaces, leveraging the power grid's inherent hierarchical nature. Experimental results indicate the MARL framework's competitive performance with single-agent RL methods. We also compare different RL algorithms for lower-level agents alongside different policies for higher-order agents.

翻译：电网运行面临的最新挑战源于能源需求持续增长以及风电、光伏等不可预测可再生能源的并网。尽管强化学习（RL）通过母线投切、线路开断等拓扑操作在电网管理方面展现出潜力，但高效处理电网规模扩大带来的庞大动作空间至关重要。本文提出一种面向大规模动作空间的分层多智能体强化学习（MARL）框架，该框架充分利用了电网固有的分层特性。实验结果表明，该MARL框架的性能可与单智能体RL方法相媲美。我们还比较了不同RL算法在下层智能体中的应用表现，以及上层智能体采用的不同策略。