The growing adoption of mmWave frequency bands to realize the full potential of 5G, turns beamforming into a key enabler for current and next-generation wireless technologies. Many mmWave networks rely on beam selection with Grid-of-Beams (GoB) approach to handle user-beam association. In beam selection with GoB, users select the appropriate beam from a set of pre-defined beams and the overhead during the beam selection process is a common challenge in this area. In this paper, we propose an Advantage Actor Critic (A2C) learning-based framework to improve the GoB and the beam selection process, as well as optimize transmission power in a mmWave network. The proposed beam selection technique allows performance improvement while considering transmission power improves Energy Efficiency (EE) and ensures the coverage is maintained in the network. We further investigate how the proposed algorithm can be deployed in a Service Management and Orchestration (SMO) platform. Our simulations show that A2C-based joint optimization of beam selection and transmission power is more effective than using Equally Spaced Beams (ESB) and fixed power strategy, or optimization of beam selection and transmission power disjointly. Compared to the ESB and fixed transmission power strategy, the proposed approach achieves more than twice the average EE in the scenarios under test and is closer to the maximum theoretical EE.

翻译：随着5G毫米波频段的广泛应用以充分发挥其潜力，波束成形技术成为当前及下一代无线技术的关键使能要素。众多毫米波网络采用基于栅格波束（GoB）的波束选择方法实现用户-波束关联。在GoB波束选择过程中，用户需从预定义波束集中选取合适波束，而波束选择过程的信令开销是该领域面临的普遍挑战。本文提出一种基于优势演员-评论家（A2C）学习的框架，用于改进GoB波束选择流程，并优化毫米波网络的发射功率。所提波束选择技术既能提升性能，又通过优化发射功率提高了能量效率（EE），同时确保网络覆盖。我们进一步研究了该算法在服务管理与编排（SMO）平台上的部署方案。仿真结果表明，基于A2C的波束选择与发射功率联合优化方法，相较于等间隔波束（ESB）固定功率策略，或对波束选择与发射功率进行分离优化的方案更具优势。与ESB固定发射功率策略相比，所提方法在测试场景下的平均能量效率提升超过两倍，且更接近理论最大能效值。