With the rapid growth of mobile data traffic, the shortage of radio spectrum resource has become increasingly prominent. Millimeter wave (mmWave) small cells can be densely deployed in macro cells to improve network capacity and spectrum utilization. Such a network architecture is referred to as mmWave heterogeneous cellular networks (HetNets). Compared with the traditional wired backhaul, The integrated access and backhaul (IAB) architecture with wireless backhaul is more flexible and cost-effective for mmWave HetNets. However, the imbalance of throughput between the access and backhaul links will constrain the total system throughput. Consequently, it is necessary to jointly design of radio access and backhaul link. In this paper, we study the joint optimization of user association and backhaul resource allocation in mmWave HetNets, where different mmWave bands are adopted by the access and backhaul links. Considering the non-convex and combinatorial characteristics of the optimization problem and the dynamic nature of the mmWave link, we propose a multi-agent deep reinforcement learning (MADRL) based scheme to maximize the long-term total link throughput of the network. The simulation results show that the scheme can not only adjust user association and backhaul resource allocation strategy according to the dynamics in the access link state, but also effectively improve the link throughput under different system configurations.

翻译：随着移动数据流量的快速增长，无线电频谱资源短缺问题日益突出。毫米波小蜂窝可密集部署于宏蜂窝内，以提升网络容量和频谱利用率。这种网络架构被称为毫米波异构蜂窝网络。与传统的有线回传相比，采用无线回传的集成接入与回传（IAB）架构在毫米波异构蜂窝网络中具有更高的灵活性和成本效益。然而，接入链路与回传链路之间的吞吐量不均衡会制约系统总吞吐量。因此，有必要对无线接入和回传链路进行联合设计。本文研究了毫米波异构蜂窝网络中用户关联与回传资源分配的联合优化问题，其中接入链路和回传链路采用不同的毫米波频段。考虑到优化问题的非凸组合特性以及毫米波链路的动态性，我们提出了一种基于多智能体深度强化学习（MADRL）的方案，以最大化网络的长期总链路吞吐量。仿真结果表明，该方案不仅能根据接入链路状态的动态性调整用户关联和回传资源分配策略，还能在不同系统配置下有效提升链路吞吐量。