This paper proposes a novel framework that leverages large language models (LLMs) to automate curriculum design, thereby enhancing the application of reinforcement learning (RL) in mobile networks. As mobile networks evolve towards the 6G era, managing their increasing complexity and dynamic nature poses significant challenges. Conventional RL approaches often suffer from slow convergence and poor generalization due to conflicting objectives and the large state and action spaces associated with mobile networks. To address these shortcomings, we introduce curriculum learning, a method that systematically exposes the RL agent to progressively challenging tasks, improving convergence and generalization. However, curriculum design typically requires extensive domain knowledge and manual human effort. Our framework mitigates this by utilizing the generative capabilities of LLMs to automate the curriculum design process, significantly reducing human effort while improving the RL agent's convergence and performance. We deploy our approach within a simulated mobile network environment and demonstrate improved RL convergence rates, generalization to unseen scenarios, and overall performance enhancements. As a case study, we consider autonomous coordination and user association in mobile networks. Our obtained results highlight the potential of combining LLM-based curriculum generation with RL for managing next-generation wireless networks, marking a significant step towards fully autonomous network operations.

翻译：本文提出了一种新颖的框架，该框架利用大型语言模型（LLMs）来自动化课程设计，从而增强强化学习（RL）在移动网络中的应用。随着移动网络向6G时代演进，管理其日益增长的复杂性和动态特性带来了重大挑战。传统的RL方法由于目标冲突以及与移动网络相关的大规模状态和动作空间，常常存在收敛速度慢和泛化能力差的问题。为了解决这些不足，我们引入了课程学习，这是一种让RL智能体系统地接触逐渐增加难度的任务，以提高其收敛性和泛化能力的方法。然而，课程设计通常需要广泛的领域知识和大量的人工投入。我们的框架通过利用LLMs的生成能力来自动化课程设计过程，显著减少了人力投入，同时提高了RL智能体的收敛性和性能。我们在一个模拟的移动网络环境中部署了我们的方法，并展示了改进的RL收敛速度、对未见场景的泛化能力以及整体性能的提升。作为一个案例研究，我们考虑了移动网络中的自主协调和用户关联。我们获得的结果突显了将基于LLM的课程生成与RL相结合用于管理下一代无线网络的潜力，标志着向完全自主的网络运营迈出了重要一步。