Deep reinforcement learning (DRL) has been increasingly employed to handle the dynamic and complex resource management in network slicing. The deployment of DRL policies in real networks, however, is complicated by heterogeneous cell conditions. In this paper, we propose a novel transfer learning (TL) aided multi-agent deep reinforcement learning (MADRL) approach with inter-agent similarity analysis for inter-cell inter-slice resource partitioning. First, we design a coordinated MADRL method with information sharing to intelligently partition resource to slices and manage inter-cell interference. Second, we propose an integrated TL method to transfer the learned DRL policies among different local agents for accelerating the policy deployment. The method is composed of a new domain and task similarity measurement approach and a new knowledge transfer approach, which resolves the problem of from whom to transfer and how to transfer. We evaluated the proposed solution with extensive simulations in a system-level simulator and show that our approach outperforms the state-of-the-art solutions in terms of performance, convergence speed and sample efficiency. Moreover, by applying TL, we achieve an additional gain over 27% higher than the coordinate MADRL approach without TL.

翻译：深度强化学习（DRL）已被越来越多地用于处理网络切片中动态且复杂的资源管理问题。然而，由于不同蜂窝小区条件的异构性，在实际网络中部署DRL策略变得复杂。本文提出一种新颖的、基于迁移学习（TL）辅助的多智能体深度强化学习（MADRL）方法，该方法通过智能体间相似性分析实现小区间与切片间的资源划分。首先，我们设计了一种带有信息共享的协调式MADRL方法，用于智能地分配切片资源并管理小区间干扰。其次，提出一种集成式迁移学习方法，将学习到的DRL策略在不同本地智能体间进行迁移，以加速策略部署。该方法包括一种新的域与任务相似度度量方法以及一种新的知识迁移方法，解决了从何处迁移以及如何迁移的问题。我们通过系统级仿真器进行了大量仿真评估，结果表明，本方法在性能、收敛速度与样本效率方面均优于现有最优方案。此外，应用迁移学习后，相比无迁移的协调式MADRL方法，我们获得了超过27%的额外增益。