Future generations of mobile networks are expected to contain more and more antennas with growing complexity and more parameters. Optimizing these parameters is necessary for ensuring the good performance of the network. The scale of mobile networks makes it challenging to optimize antenna parameters using manual intervention or hand-engineered strategies. Reinforcement learning is a promising technique to address this challenge but existing methods often use local optimizations to scale to large network deployments. We propose a new multi-agent reinforcement learning algorithm to optimize mobile network configurations globally. By using a value decomposition approach, our algorithm can be trained from a global reward function instead of relying on an ad-hoc decomposition of the network performance across the different cells. The algorithm uses a graph neural network architecture which generalizes to different network topologies and learns coordination behaviors. We empirically demonstrate the performance of the algorithm on an antenna tilt tuning problem and a joint tilt and power control problem in a simulated environment.

翻译：未来移动网络预计将包含越来越多天线，其复杂度与参数数量持续增长。优化这些参数对于保障网络性能至关重要。移动网络的庞大规模使得人工干预或手工设计策略难以有效优化天线参数。强化学习是应对这一挑战的有前景的技术，但现有方法常采用局部优化策略以扩展至大规模网络部署。我们提出一种新型多智能体强化学习算法，用于全局优化移动网络配置。通过采用值分解方法，该算法可直接基于全局奖励函数进行训练，无需依赖跨不同小区的网络性能特设分解。算法采用图神经网络架构，可泛化至不同网络拓扑并学习协同行为。我们在模拟环境中通过天线倾角调优问题以及倾角与功率联合控制问题，实证验证了该算法的性能。