Scalable load balancing algorithms are of great interest in cloud networks and data centers, necessitating the use of tractable techniques to compute optimal load balancing policies for good performance. However, most existing scalable techniques, especially asymptotically scaling methods based on mean field theory, have not been able to model large queueing networks with strong locality. Meanwhile, general multi-agent reinforcement learning techniques can be hard to scale and usually lack a theoretical foundation. In this work, we address this challenge by leveraging recent advances in sparse mean field theory to learn a near-optimal load balancing policy in sparsely connected queueing networks in a tractable manner, which may be preferable to global approaches in terms of wireless communication overhead. Importantly, we obtain a general load balancing framework for a large class of sparse bounded-degree wireless topologies. By formulating a novel mean field control problem in the context of graphs with bounded degree, we reduce the otherwise difficult multi-agent problem to a single-agent problem. Theoretically, the approach is justified by approximation guarantees. Empirically, the proposed methodology performs well on several realistic and scalable wireless network topologies as compared to a number of well-known load balancing heuristics and existing scalable multi-agent reinforcement learning methods.

翻译：可扩展的负载均衡算法在云网络和数据中心中备受关注，需要采用易处理的技术来计算最优负载均衡策略以获得良好性能。然而，现有的大多数可扩展技术，尤其是基于平均场理论的渐近缩放方法，尚无法对具有强局部性的大规模排队网络进行建模。同时，通用的多智能体强化学习技术难以扩展，且通常缺乏理论基础。在本工作中，我们通过利用稀疏平均场理论的最新进展，以易处理的方式在稀疏连接的排队网络中学习近似最优的负载均衡策略，这从无线通信开销角度而言可能优于全局方法。重要的是，我们为一大类稀疏有界度无线拓扑结构建立了一个通用负载均衡框架。通过在有限度图背景下提出一个新颖的平均场控制问题，我们将原本困难的多智能体问题简化为单智能体问题。理论上，该方法具有近似保证的合理性。实验表明，与多种知名负载均衡启发式算法及现有可扩展多智能体强化学习方法相比，所提方法在多个现实且可扩展的无线网络拓扑上表现良好。