With the broader usage and highly successful development of Large Language Models (LLMs), there has been a growth of interest and demand for applying LLMs to autonomous driving technology. Driven by their natural language understanding and reasoning ability, LLMs have the potential to enhance various aspects of autonomous driving systems, from perception and scene understanding to language interaction and decision-making. In this paper, we first introduce the novel concept of designing LLMs for autonomous driving (LLM4AD). Then, we propose a comprehensive benchmark for evaluating the instruction-following abilities of LLM4AD in simulation. Furthermore, we conduct a series of experiments on real-world vehicle platforms, thoroughly evaluating the performance and potential of our LLM4AD systems. Finally, we envision the main challenges of LLM4AD, including latency, deployment, security and privacy, safety, trust and transparency, and personalization. Our research highlights the significant potential of LLMs to enhance various aspects of autonomous vehicle technology, from perception and scene understanding to language interaction and decision-making.

翻译：随着大语言模型（LLM）的广泛应用和高度成功的发展，将LLM应用于自动驾驶技术的兴趣和需求日益增长。凭借其自然语言理解和推理能力，LLM有潜力增强自动驾驶系统的各个方面，从感知和场景理解到语言交互与决策。本文首先介绍了为自动驾驶设计大语言模型（LLM4AD）这一新颖概念。随后，我们提出了一个全面的基准，用于在仿真环境中评估LLM4AD的指令遵循能力。此外，我们在真实车辆平台上进行了一系列实验，全面评估了我们LLM4AD系统的性能与潜力。最后，我们展望了LLM4AD面临的主要挑战，包括延迟、部署、安全与隐私、安全性、信任与透明度以及个性化。我们的研究突显了LLM在增强自动驾驶车辆技术各方面（从感知和场景理解到语言交互与决策）的巨大潜力。