This paper proposes a multiagent based bi-level operation framework for the low-carbon demand management in distribution networks considering the carbon emission allowance on the demand side. In the upper level, the aggregate load agents optimize the control signals for various types of loads to maximize the profits; in the lower level, the distribution network operator makes optimal dispatching decisions to minimize the operational costs and calculates the distribution locational marginal price and carbon intensity. The distributed flexible load agent has only incomplete information of the distribution network and cooperates with other agents using networked communication. Finally, the problem is formulated into a networked multi-agent constrained Markov decision process, which is solved using a safe reinforcement learning algorithm called consensus multi-agent constrained policy optimization considering the carbon emission allowance for each agent. Case studies with the IEEE 33-bus and 123-bus distribution network systems demonstrate the effectiveness of the proposed approach, in terms of satisfying the carbon emission constraint on demand side, ensuring the safe operation of the distribution network and preserving privacy of both sides.

翻译：本文提出了一种基于多智能体的双层运行框架，用于配电网低碳需求管理，同时考虑需求侧的碳排放配额。在上层，聚合负荷智能体优化各类负荷的控制信号以最大化收益；在下层，配电网运营商做出最优调度决策以最小化运营成本，并计算配电网节点边际电价与碳排放强度。分布式柔性负荷智能体仅掌握配电网的不完全信息，并通过网络化通信与其他智能体协作。最终，该问题被建模为网络化多智能体约束马尔可夫决策过程，并通过一种考虑各智能体碳排放配额的安全强化学习算法——共识多智能体约束策略优化进行求解。基于IEEE 33节点与123节点配电网系统的算例分析表明，所提方法在满足需求侧碳排放约束、保障配电网安全运行及保护双方隐私方面具有有效性。