Large language models (LLMs) and multimodal large language models (MLLMs) have shown excellent general capabilities, even exhibiting adaptability in many professional domains such as law, economics, transportation, and medicine. Currently, many domain-specific benchmarks have been proposed to verify the performance of (M)LLMs in specific fields. Among various domains, transportation plays a crucial role in modern society as it impacts the economy, the environment, and the quality of life for billions of people. However, it is unclear how much traffic knowledge (M)LLMs possess and whether they can reliably perform transportation-related tasks. To address this gap, we propose TransportationGames, a carefully designed and thorough evaluation benchmark for assessing (M)LLMs in the transportation domain. By comprehensively considering the applications in real-world scenarios and referring to the first three levels in Bloom's Taxonomy, we test the performance of various (M)LLMs in memorizing, understanding, and applying transportation knowledge by the selected tasks. The experimental results show that although some models perform well in some tasks, there is still much room for improvement overall. We hope the release of TransportationGames can serve as a foundation for future research, thereby accelerating the implementation and application of (M)LLMs in the transportation domain.

翻译：大语言模型（LLMs）与多模态大语言模型（MLLMs）展现出卓越的通用能力，在法学、经济学、交通运输、医学等多个专业领域亦表现出适应性。当前，众多领域专用基准被提出以验证（多模态）大语言模型在特定领域中的性能。其中，交通运输作为影响经济、环境及数十亿人生活质量的现代社会的核心要素，其重要性不言而喻。然而，目前尚不明确（多模态）大语言模型究竟掌握多少交通知识，以及能否可靠执行交通相关任务。为填补这一空白，我们提出了"运输游戏"——一个经精心设计且评估全面的基准，用于衡量（多模态）大语言模型在交通领域的表现。通过综合考量真实场景应用，并参照布鲁姆认知分类学的前三个层次，我们设计了涵盖记忆、理解与应用交通知识的任务，系统检测了多种（多模态）大语言模型的性能。实验结果表明，尽管部分模型在特定任务中表现良好，但总体仍存在显著提升空间。我们期望"运输游戏"的发布能为未来研究奠定基础，从而加速（多模态）大语言模型在交通领域的落地与应用。