Numerous solutions are proposed for the Traffic Signal Control (TSC) tasks aiming to provide efficient transportation and mitigate congestion waste. In recent, promising results have been attained by Reinforcement Learning (RL) methods through trial and error in simulators, bringing confidence in solving cities' congestion headaches. However, there still exist performance gaps when simulator-trained policies are deployed to the real world. This issue is mainly introduced by the system dynamic difference between the training simulator and the real-world environments. The Large Language Models (LLMs) are trained on mass knowledge and proved to be equipped with astonishing inference abilities. In this work, we leverage LLMs to understand and profile the system dynamics by a prompt-based grounded action transformation. Accepting the cloze prompt template, and then filling in the answer based on accessible context, the pre-trained LLM's inference ability is exploited and applied to understand how weather conditions, traffic states, and road types influence traffic dynamics, being aware of this, the policies' action is taken and grounded based on realistic dynamics, thus help the agent learn a more realistic policy. We conduct experiments using DQN to show the effectiveness of the proposed PromptGAT's ability in mitigating the performance gap from simulation to reality (sim-to-real).

翻译：针对交通信号控制任务，研究者提出了众多解决方案，旨在实现高效交通并缓解拥堵浪费。近年来，强化学习方法通过在模拟器中的试错机制取得了令人瞩目的成果，为解决城市交通拥堵难题带来了信心。然而，当在模拟器上训练的策略部署到现实世界时，仍存在性能差距。这一问题主要由训练模拟器与真实环境之间的系统动态差异导致。大语言模型经过海量知识训练，被证明具备惊人的推理能力。本文利用大语言模型通过基于提示的具身动作转换来理解并刻画系统动态。通过接受完形填空式提示模板，并基于可获取的上下文填充答案，我们挖掘并应用预训练大语言模型的推理能力，使其理解天气状况、交通状态和道路类型如何影响交通动态。基于此认知，策略的动作根据现实动态被采纳并具身化，从而帮助智能体学习更贴合现实的策略。我们采用DQN进行实验，验证了所提出的PromptGAT在缩小从仿真到现实（sim-to-real）的性能差距方面的有效性。