Connected and automated vehicles (CAVs) are considered a potential solution for future transportation challenges, aiming to develop systems that are efficient, safe, and environmentally friendly. However, CAV control presents significant challenges due to the complexity of interconnectivity and coordination required among vehicles. Multi-agent reinforcement learning (MARL), which has shown notable advancements in addressing complex problems in autonomous driving, robotics, and human-vehicle interaction, emerges as a promising tool to enhance CAV capabilities. Despite its potential, there is a notable absence of current reviews on mainstream MARL algorithms for CAVs. To fill this gap, this paper offers a comprehensive review of MARL's application in CAV control. The paper begins with an introduction to MARL, explaining its unique advantages in handling complex and multi-agent scenarios. It then presents a detailed survey of MARL applications across various control dimensions for CAVs, including critical scenarios such as platooning control, lane-changing, and unsignalized intersections. Additionally, the paper reviews prominent simulation platforms essential for developing and testing MARL algorithms. Lastly, it examines the current challenges in deploying MARL for CAV control, including macro-micro optimization, communication, mixed traffic, and sim-to-real challenges. Potential solutions discussed include hierarchical MARL, decentralized MARL, adaptive interactions, and offline MARL.

翻译：网联自动驾驶车辆（CAV）被视为应对未来交通挑战的潜在解决方案，其目标是开发高效、安全且环保的交通系统。然而，由于车辆间所需的互联互通与协调机制极为复杂，CAV控制面临着重大挑战。多智能体强化学习（MARL）在自动驾驶、机器人学及人车交互等领域处理复杂问题方面已展现出显著进展，成为提升CAV能力的有效工具。尽管潜力巨大，目前尚缺乏针对CAV的主流MARL算法的系统性综述。为填补这一空白，本文对MARL在CAV控制中的应用进行了全面回顾。文章首先介绍了MARL的基本原理，阐释了其在处理复杂多智能体场景中的独特优势；随后详细综述了MARL在CAV多维度控制中的应用，涵盖队列控制、变道及无信号交叉口等关键场景；此外，本文还梳理了MARL算法开发与测试所必需的主流仿真平台；最后，探讨了MARL应用于CAV控制当前面临的挑战，包括宏观-微观协同优化、通信机制、混合交通流及仿真到现实的迁移等问题。文中讨论的潜在解决方案涵盖分层MARL、去中心化MARL、自适应交互机制以及离线MARL等方向。