Coordinating large populations of grid-edge devices requires learning methods that remain fully decentralised in deployment while still respecting three-phase AC distribution-network physics. This paper proposes gradient-based multi-agent proximal learning (GradMAP) to address this challenge. GradMAP trains independent neural-network policies for each agent without any parameter sharing, and each agent uses only its own local observation for online decision-making without communication. During offline training, GradMAP embeds a differentiable three-phase AC power-flow model in a primal-dual learning loop and uses implicit differentiation to propagate exact network-constraint violations to update the policy parameters. To speed up training, GradMAP reuses expensive environment gradients through a proximal surrogate within a trust region defined in the more direct policy-output (action) space, instead of the probability distribution space used in other works, such as PPO. In case studies with 1,000 agents managing batteries, heat pumps, and controllable generators on the IEEE 123-bus feeder, GradMAP learns decentralised policies that minimise three-phase AC load-flow constraint violations within 15 minutes of training on a single workstation-class NVIDIA RTX PRO 5000 Blackwell 48GB GPU. This is a 3--5x training speed-up over gradient-based self-supervised learning benchmarks and substantially better training efficiency than multi-agent reinforcement-learning benchmarks. In out-of-sample tests, GradMAP also delivers among the lowest operating cost and constraint violations.

翻译：[translated abstract in Chinese] 协调电网边缘设备的大规模群体需要一种学习方法，在部署时保持完全去中心化，同时仍需遵守三相交流配电网的物理约束。本文提出基于梯度的多智能体近端学习（GradMAP）以应对这一挑战。GradMAP为每个智能体训练独立的神经网络策略，无需参数共享，且每个智能体仅利用自身局部观测进行在线决策，无需通信。在离线训练阶段，GradMAP将可微分的三相交流潮流模型嵌入原始-对偶学习循环中，利用隐式微分传播精确的网络约束违反信息以更新策略参数。为加速训练，GradMAP通过近端代理在更直接的策略输出（动作）空间（而非其他方法如PPO所使用的概率分布空间）所定义的信任域内，复用昂贵的环境梯度。在涉及IEEE 123节点馈线上1000个智能体管理电池、热泵和可控发电机的案例研究中，GradMAP在单台工作站级NVIDIA RTX PRO 5000 Blackwell 48GB GPU上，仅需15分钟训练即可学习到最小化三相交流潮流约束违反的去中心化策略。相较于基于梯度的自监督学习基准，其训练速度提升3-5倍，且训练效率显著优于多智能体强化学习基准。在样本外测试中，GradMAP同时实现了最低的运行成本和约束违反量。