Microgrids (MG) are anticipated to be important players in the future smart grid. For proper operation of MGs an Energy Management System (EMS) is essential. The EMS of an MG could be rather complicated when renewable energy resources (RER), energy storage system (ESS) and demand side management (DSM) need to be orchestrated. Furthermore, these systems may belong to different entities and competition may exist between them. Nash equilibrium is most commonly used for coordination of such entities however the convergence and existence of Nash equilibrium can not always be guaranteed. To this end, we use the correlated equilibrium to coordinate agents, whose convergence can be guaranteed. In this paper, we build an energy trading model based on mid-market rate, and propose a correlated Q-learning (CEQ) algorithm to maximize the revenue of each agent. Our results show that CEQ is able to balance the revenue of agents without harming total benefit. In addition, compared with Q-learning without correlation, CEQ could save 19.3% cost for the DSM agent and 44.2% more benefits for the ESS agent.

翻译：微电网(MG)预计将在未来智能电网中成为重要角色。 要使MGs适当运行一个能源管理系统(EMS)至关重要。 当需要调整可再生能源资源(RER)、能源储存系统和需求方管理(DSM)时,MGs的环管系统可能相当复杂。 此外,这些系统可能属于不同实体,彼此之间可能存在竞争。 Nash 平衡最常用于协调这些实体,但Nash 平衡的趋同和存在并不总是得到保证。 为此,我们利用相关平衡来协调可以保证其趋同的代理商。 在本文中,我们建立一个基于中市利率的能源交易模式,并提出一个相关的Q-学习算法,以最大限度地增加每个代理商的收入。我们的结果表明,CEQ能够平衡各种代理商的收入,而不会损害总的利益。 此外,与Q-学习相比,CEQ可以节省DSM代理商19.3%的费用,使ESP代理商的收益增加44.2%。