With the broader usage and highly successful development of Large Language Models (LLMs), there has been a growth of interest and demand for applying LLMs to autonomous driving technology. Driven by their natural language understanding and reasoning ability, LLMs have the potential to enhance various aspects of autonomous driving systems, from perception and scene understanding to language interaction and decision-making. In this paper, we first introduce novel concepts and approaches to designing LLMs for autonomous driving (LLM4AD). Then, we propose a comprehensive benchmark for evaluating the instruction-following abilities of LLMs within the autonomous driving domain. Furthermore, we conduct a series of experiments on both simulation and real-world vehicle platforms, thoroughly evaluating the performance and potential of our LLM4AD systems. Our research highlights the significant potential of LLMs to enhance various aspects of autonomous vehicle technology, from perception and scene understanding to language interaction and decision-making.

翻译：随着大型语言模型（LLM）的广泛应用和高度成功发展，将LLM应用于自动驾驶技术的兴趣和需求日益增长。凭借其自然语言理解和推理能力，LLM有潜力提升自动驾驶系统的各个方面，从感知与场景理解到语言交互与决策制定。本文首先介绍了为自动驾驶设计大型语言模型（LLM4AD）的新颖概念与方法。随后，我们提出了一个全面的基准，用于评估LLM在自动驾驶领域内的指令遵循能力。此外，我们在仿真和真实车辆平台上进行了一系列实验，全面评估了我们LLM4AD系统的性能与潜力。我们的研究突显了LLM在提升自动驾驶车辆技术多个方面（从感知与场景理解到语言交互与决策制定）的巨大潜力。