Multi-agent Reinforcement Learning (MARL) has gained wide attention in recent years and has made progress in various fields. Specifically, cooperative MARL focuses on training a team of agents to cooperatively achieve tasks that are difficult for a single agent to handle. It has shown great potential in applications such as path planning, autonomous driving, active voltage control, and dynamic algorithm configuration. One of the research focuses in the field of cooperative MARL is how to improve the coordination efficiency of the system, while research work has mainly been conducted in simple, static, and closed environment settings. To promote the application of artificial intelligence in real-world, some research has begun to explore multi-agent coordination in open environments. These works have made progress in exploring and researching the environments where important factors might change. However, the mainstream work still lacks a comprehensive review of the research direction. In this paper, starting from the concept of reinforcement learning, we subsequently introduce multi-agent systems (MAS), cooperative MARL, typical methods, and test environments. Then, we summarize the research work of cooperative MARL from closed to open environments, extract multiple research directions, and introduce typical works. Finally, we summarize the strengths and weaknesses of the current research, and look forward to the future development direction and research problems in cooperative MARL in open environments.

翻译：多智能体强化学习（MARL）近年来受到广泛关注，并在多个领域取得进展。其中，协同MARL专注于训练智能体团队协作完成单智能体难以胜任的任务，在路径规划、自动驾驶、主动电压控制和动态算法配置等应用中展现出巨大潜力。该领域的研究重点之一在于提升系统协调效率，而现有研究主要集中于简单、静态且封闭的环境设定。为促进人工智能在现实世界中的应用，部分研究已开始探索开放环境下的多智能体协同问题。这些工作虽在探究重要因素可能变化的环境方面取得进展，但主流研究仍缺乏对该方向的系统性综述。本文从强化学习概念出发，依次介绍多智能体系统（MAS）、协同MARL、典型方法与测试环境，并综述从封闭环境到开放环境的协同MARL研究进展，提炼出多个研究方向及代表性工作。最后，总结当前研究的优势与不足，展望开放环境下协同MARL的未来发展方向与研究问题。