With the advancement of artificial intelligence technology, the automation of network management, also known as Autonomous Driving Networks (ADN), is gaining widespread attention. The network management has shifted from traditional homogeneity and centralization to heterogeneity and decentralization. Multi-agent deep reinforcement learning (MADRL) allows agents to make decisions based on local observations independently. This approach is in line with the needs of automation and has garnered significant attention from academia and industry. In a distributed environment, information interaction between agents can effectively address the non-stationarity problem of multiple agents and promote cooperation. Therefore, in this survey, we first examined the application of MADRL in network management, including specific application fields such as traffic engineering, wireless network access, power control, and network security. Then, we conducted a detailed analysis of communication behavior between agents, including communication schemes, communication content construction, communication object selection, message processing, and communication constraints. Finally, we discussed the open issues and future research directions of agent communication in MADRL for future network management and ADN applications.
翻译:随着人工智能技术的进步,网络管理自动化(亦称为自动驾驶网络)正获得广泛关注。网络管理已从传统的同质化与集中式转向异质化与分布式。多智能体深度强化学习允许智能体基于局部观测独立做出决策,该方法契合自动化需求,已引起学术界与工业界的广泛重视。在分布式环境中,智能体间的信息交互能有效应对多智能体的非平稳性问题并促进协作。因此,本文首先梳理了多智能体深度强化学习在网络管理中的应用,涵盖流量工程、无线网络接入、功率控制及网络安全等具体应用领域;继而详细分析了智能体间的通信行为,包括通信方案设计、通信内容构建、通信对象选择、消息处理机制及通信约束条件;最后探讨了面向未来网络管理与自动驾驶网络应用时,多智能体深度强化学习中智能体通信面临的开放性问题及未来研究方向。