Over the years, reinforcement learning has emerged as a popular approach to develop signal control and vehicle platooning strategies either independently or in a hierarchical way. However, jointly controlling both in real-time to alleviate traffic congestion presents new challenges, such as the inherent physical and behavioral heterogeneity between signal control and platooning, as well as coordination between them. This paper proposes an innovative solution to tackle these challenges based on heterogeneous graph multi-agent reinforcement learning and traffic theories. Our approach involves: 1) designing platoon and signal control as distinct reinforcement learning agents with their own set of observations, actions, and reward functions to optimize traffic flow; 2) designing coordination by incorporating graph neural networks within multi-agent reinforcement learning to facilitate seamless information exchange among agents on a regional scale. We evaluate our approach through SUMO simulation, which shows a convergent result in terms of various transportation metrics and better performance over sole signal or platooning control.

翻译：多年来，强化学习已成为一种流行方法，用于独立或分层地制定信号控制与车辆编队策略。然而，实时联合控制两者以缓解交通拥堵带来了新挑战，例如信号控制与编队之间固有的物理与行为异质性，以及两者间的协同问题。本文提出一种基于异构图多智能体强化学习与交通理论的创新解决方案。我们的方法包括：1）将编队与信号控制设计为具有各自观测空间、动作空间及奖励函数的独立强化学习智能体，以优化交通流；2）通过在多智能体强化学习中融入图神经网络设计协同机制，促进区域尺度上智能体间的无缝信息交换。我们通过SUMO仿真评估了该方法，结果表明其在多种交通指标上实现了收敛性能，并且优于单独使用信号控制或编队控制的方案。