Cooperative Adaptive Cruise Control (CACC) plays a pivotal role in enhancing traffic efficiency and safety in Connected and Autonomous Vehicles (CAVs). Reinforcement Learning (RL) has proven effective in optimizing complex decision-making processes in CACC, leading to improved system performance and adaptability. Among RL approaches, Multi-Agent Reinforcement Learning (MARL) has shown remarkable potential by enabling coordinated actions among multiple CAVs through Centralized Training with Decentralized Execution (CTDE). However, MARL often faces scalability issues, particularly when CACC vehicles suddenly join or leave the platoon, resulting in performance degradation. To address these challenges, we propose Communication-Aware Reinforcement Learning (CA-RL). CA-RL includes a communication-aware module that extracts and compresses vehicle communication information through forward and backward information transmission modules. This enables efficient cyclic information propagation within the CACC traffic flow, ensuring policy consistency and mitigating the scalability problems of MARL in CACC. Experimental results demonstrate that CA-RL significantly outperforms baseline methods in various traffic scenarios, achieving superior scalability, robustness, and overall system performance while maintaining reliable performance despite changes in the number of participating vehicles.

翻译：协同自适应巡航控制（CACC）在提升网联自动驾驶车辆（CAV）的交通效率与安全性方面发挥着关键作用。强化学习（RL）已被证明能有效优化CACC中的复杂决策过程，从而提升系统性能与适应性。在多智能体强化学习（MARL）方法中，通过集中训练分散执行（CTDE）机制实现多辆CAV间的协同行动，已展现出显著潜力。然而，MARL常面临可扩展性问题，尤其在CACC车辆突然加入或离开车队时，会导致性能下降。为应对这些挑战，我们提出了通信感知强化学习（CA-RL）。CA-RL包含一个通信感知模块，该模块通过前向与后向信息传输模块提取并压缩车辆通信信息。这使得CACC交通流内能够实现高效循环信息传播，从而确保策略一致性并缓解MARL在CACC中的可扩展性问题。实验结果表明，CA-RL在多种交通场景中均显著优于基线方法，在参与车辆数量变化的情况下仍保持可靠性能，并实现了更优的可扩展性、鲁棒性及整体系统性能。