Traffic congestion is a persistent problem in our society. Previous methods for traffic control have proven futile in alleviating current congestion levels leading researchers to explore ideas with robot vehicles given the increased emergence of vehicles with different levels of autonomy on our roads. This gives rise to mixed traffic control, where robot vehicles regulate human-driven vehicles through reinforcement learning (RL). However, most existing studies use precise observations that require domain expertise and hand engineering for each road network's observation space. Additionally, precise observations use global information, such as environment outflow, and local information, i.e., vehicle positions and velocities. Obtaining this information requires updating existing road infrastructure with vast sensor environments and communication to potentially unwilling human drivers. We consider image observations, a modality that has not been extensively explored for mixed traffic control via RL, as the alternative: 1) images do not require a complete re-imagination of the observation space from environment to environment; 2) images are ubiquitous through satellite imagery, in-car camera systems, and traffic monitoring systems; and 3) images only require communication to equipment. In this work, we show robot vehicles using image observations can achieve competitive performance to using precise information on environments, including ring, figure eight, intersection, merge, and bottleneck. In certain scenarios, our approach even outperforms using precision observations, e.g., up to 8% increase in average vehicle velocity in the merge environment, despite only using local traffic information as opposed to global traffic information.

翻译：交通拥堵是当今社会持续存在的难题。先前交通控制方法在缓解当前拥堵水平方面效果有限，促使研究人员探索利用机器人车辆——鉴于道路上不同自动驾驶等级车辆的日益普及——来提出解决方案。由此催生了混合交通控制，即机器人车辆通过强化学习调节人类驾驶车辆。然而，现有研究大多依赖精确观测，这需要对每个路网的观测空间进行专业领域知识和手工工程化处理。此外，精确观测需使用环境流出量等全局信息以及车辆位置、速度等局部信息。获取这些信息需要更新现有道路基础设施，配备大规模传感器环境，并向可能存在抵触心理的人类驾驶员传输数据。我们考虑采用图像观测这一在混合交通控制强化学习中尚未广泛探索的替代方案：1）图像无需根据环境变化完全重构观测空间；2）通过卫星影像、车载摄像头系统和交通监控设备可广泛获取图像信息；3）图像仅需与设备的通信。本研究表明，使用图像观测的机器人车辆在环形、8字型、交叉口、合流及瓶颈等场景中，其性能可与采用精确信息的方法相媲美。在某些场景下，例如合流环境中，尽管仅使用局部交通信息而非全局信息，我们的方法在平均车速上仍能超越使用精确观测的方法，最高提升达8%。