In cellular networks, User Equipment (UE) handoff from one Base Station (BS) to another, giving rise to the load balancing problem among the BSs. To address this problem, BSs can work collaboratively to deliver a smooth migration (or handoff) and satisfy the UEs' service requirements. This paper formulates the load balancing problem as a Markov game and proposes a Robust Multi-agent Attention Actor-Critic (Robust-MA3C) algorithm that can facilitate collaboration among the BSs (i.e., agents). In particular, to solve the Markov game and find a Nash equilibrium policy, we embrace the idea of adopting a nature agent to model the system uncertainty. Moreover, we utilize the self-attention mechanism, which encourages high-performance BSs to assist low-performance BSs. In addition, we consider two types of schemes, which can facilitate load balancing for both active UEs and idle UEs. We carry out extensive evaluations by simulations, and simulation results illustrate that, compared to the state-of-the-art MARL methods, Robust-\ours~scheme can improve the overall performance by up to 45%.

翻译：在蜂窝网络中，用户设备（UE）在不同基站（BS）之间切换时，会引发基站间的负载均衡问题。为解决该问题，基站可协同工作以实现平滑迁移（或切换），同时满足用户设备的服务需求。本文将负载均衡问题建模为马尔可夫博弈，并提出一种鲁棒多智能体注意力Actor-Critic（Robust-MA3C）算法，该算法能够促进基站（即智能体）之间的协作。具体而言，为求解马尔可夫博弈并找到纳什均衡策略，我们引入自然智能体概念以建模系统不确定性。同时，利用自注意力机制促进高性能基站辅助低性能基站。此外，我们考虑两种方案，分别实现活跃用户设备与空闲用户设备的负载均衡。通过大量仿真评估，结果表明：与现有最优的多智能体强化学习（MARL）方法相比，Robust-本方案可提升整体性能高达45%。