The disaggregated and hierarchical architecture of advanced RAN presents significant challenges in efficiently placing baseband functions and user plane functions in conjunction with Multi-Access Edge Computing (MEC) to accommodate diverse 5G services. Therefore, this paper proposes a novel approach NetMind, which leverages Deep Reinforcement Learning (DRL) to determine the function placement strategies in RANs with diverse topologies, aiming at minimizing power consumption. NetMind formulates the function placement problem as a maze-solving task, enabling a Markov Decision Process with standardized action space scales across different networks. Additionally, a Graph Convolutional Network (GCN) based encoding mechanism is introduced, allowing features from different networks to be aggregated into a single RL agent. That facilitates the RL agent's generalization capability and minimizes the negative impact of retraining on power consumption. In an example with three sub-networks, NetMind achieves comparable performance to traditional methods that require a dedicated DRL agent for each network, resulting in a 70% reduction in training costs. Furthermore, it demonstrates a substantial 32.76% improvement in power savings and a 41.67% increase in service stability compared to benchmarks from the existing literature.

翻译：先进的RAN（无线接入网）解耦式分层架构在结合多接入边缘计算（MEC）部署基带功能与用户面功能时面临严峻挑战，难以高效适配多样化5G业务需求。为此，本文提出创新方法NetMind，利用深度强化学习（DRL）为具有不同拓扑结构的RAN确定功能部署策略，以最小化功耗为目标。NetMind将功能部署问题形式化为迷宫求解任务，构建了标准化动作空间规模跨网络统一的马尔可夫决策过程。同时引入基于图卷积网络（GCN）的编码机制，使得不同网络的特征能够被聚合至单一强化学习智能体，从而增强智能体的泛化能力，最大限度降低重训练对功耗的负面影响。在包含三个子网络的实验案例中，NetMind实现了与传统每网络专属DRL智能体方法相当的性能，训练成本降低70%。相较于现有文献基准方法，其功耗节省显著提升32.76%，服务稳定性提高41.67%。