Reinforcement learning (RL) techniques for traffic signal control (TSC) have gained increasing popularity in recent years. However, most existing RL-based TSC methods tend to focus primarily on the RL model structure while neglecting the significance of proper traffic state representation. Furthermore, some RL-based methods heavily rely on expert-designed traffic signal phase competition. In this paper, we present a novel approach to TSC that utilizes queue length as an efficient state representation. We propose two new methods: (1) Max Queue-Length (M-QL), an optimization-based traditional method designed based on the property of queue length; and (2) AttentionLight, an RL model that employs the self-attention mechanism to capture the signal phase correlation without requiring human knowledge of phase relationships. Comprehensive experiments on multiple real-world datasets demonstrate the effectiveness of our approach: (1) the M-QL method outperforms the latest RL-based methods; (2) AttentionLight achieves a new state-of-the-art performance; and (3) our results highlight the significance of proper state representation, which is as crucial as neural network design in TSC methods. Our findings have important implications for advancing the development of more effective and efficient TSC methods. Our code is released on Github (https://github. com/LiangZhang1996/AttentionLight).

翻译：近年来，基于强化学习（RL）的交通信号控制（TSC）技术日益受到关注。然而，现有基于RL的TSC方法大多聚焦于RL模型结构本身，却忽视了恰当交通状态表示的重要性。此外，部分RL方法严重依赖专家设计的信号相位竞争机制。本文提出一种新颖的TSC方法，采用队列长度作为高效的状态表示。我们提出两种新方法：（1）最大队列长度（M-QL）——基于队列长度特性设计的传统优化方法；（2）AttentionLight——利用自注意力机制捕捉信号相位关联的RL模型，无需人工预设相位关系知识。在多个真实数据集上的综合实验验证了本方法的有效性：（1）M-QL方法性能超越最新基于RL的方法；（2）AttentionLight实现了新的最先进性能；（3）我们的结果凸显了合理状态表示的重要性，其与神经网络设计在TSC方法中具有同等关键地位。本研究对推动更高效TSC方法的发展具有重要启示意义。代码已开源至GitHub（https://github.com/LiangZhang1996/AttentionLight）。