Utilizing distributed renewable and energy storage resources in local distribution networks via peer-to-peer (P2P) energy trading has long been touted as a solution to improve energy systems' resilience and sustainability. Consumers and prosumers (those who have energy generation resources), however, do not have the expertise to engage in repeated P2P trading, and the zero-marginal costs of renewables present challenges in determining fair market prices. To address these issues, we propose multi-agent reinforcement learning (MARL) frameworks to help automate consumers' bidding and management of their solar PV and energy storage resources, under a specific P2P clearing mechanism that utilizes the so-called supply-demand ratio. In addition, we show how the MARL frameworks can integrate physical network constraints to realize voltage control, hence ensuring physical feasibility of the P2P energy trading and paving way for real-world implementations.

翻译：利用分布式可再生能源和储能资源，通过点对点能量交易在本地配电网中实现能量交互，长期以来被视为提升能源系统韧性和可持续性的解决方案。然而，消费者和产消者（拥有能源发电资源的用户）缺乏参与重复性点对点交易的专业知识，且可再生能源的零边际成本特性给公平市场定价带来了挑战。针对这些问题，我们提出了基于多智能体强化学习的框架，在采用供需比清算机制的点对点交易中，自动完成消费者的投标策略以及光伏与储能资源的管理。此外，我们展示了如何将物理网络约束融入该框架以实现电压控制，从而保障点对点能量交易的物理可行性，为实际应用铺平道路。