In networked multi-agent reinforcement learning (Networked-MARL), decentralized agents must act under local observability and constrained communication over fixed physical graphs. Existing methods often assume static neighborhoods, limiting adaptability to dynamic or heterogeneous environments. While centralized frameworks can learn dynamic graphs, their reliance on global state access and centralized infrastructure is impractical in real-world decentralized systems. We propose a stochastic graph-based policy for Networked-MARL, where each agent conditions its decision on a sampled subgraph over its local physical neighborhood. Building on this formulation, we introduce BayesG, a decentralized actor-framework that learns sparse, context-aware interaction structures via Bayesian variational inference. Each agent operates over an ego-graph and samples a latent communication mask to guide message passing and policy computation. The variational distribution is trained end-to-end alongside the policy using an evidence lower bound (ELBO) objective, enabling agents to jointly learn both interaction topology and decision-making strategies. BayesG outperforms strong MARL baselines on large-scale traffic control tasks with up to 167 agents, demonstrating superior scalability, efficiency, and performance.

翻译：在网络化多智能体强化学习（Networked-MARL）中，分散的智能体必须在局部可观测性以及固定物理图上的受限通信条件下进行决策。现有方法通常假设邻域是静态的，这限制了其对动态或异构环境的适应能力。虽然集中式框架可以学习动态图，但它们对全局状态访问和集中式基础设施的依赖在实际的分散式系统中并不实用。我们提出了一种基于随机图的策略用于Networked-MARL，其中每个智能体根据其局部物理邻域上采样的子图来条件化其决策。基于此框架，我们引入了BayesG，一种分散式的行动者框架，它通过贝叶斯变分推断学习稀疏的、上下文感知的交互结构。每个智能体在一个自我图上运行，并采样一个潜在的通信掩码来指导消息传递和策略计算。变分分布与策略一起通过证据下界（ELBO）目标进行端到端训练，使智能体能够共同学习交互拓扑和决策策略。在涉及多达167个智能体的大规模交通控制任务中，BayesG优于强大的MARL基线，展现出卓越的可扩展性、效率和性能。