Large Language Models (LLMs), AI models trained on massive text corpora with remarkable language understanding and generation capabilities, are transforming the field of Autonomous Driving (AD). As AD systems evolve from rule-based and optimization-based methods to learning-based techniques like deep reinforcement learning, they are now poised to embrace a third and more advanced category: knowledge-based AD empowered by LLMs. This shift promises to bring AD closer to human-like AD. However, integrating LLMs into AD systems poses challenges in real-time inference, safety assurance, and deployment costs. This survey provides a comprehensive and critical review of recent progress in leveraging LLMs for AD, focusing on their applications in modular AD pipelines and end-to-end AD systems. We highlight key advancements, identify pressing challenges, and propose promising research directions to bridge the gap between LLMs and AD, thereby facilitating the development of more human-like AD systems. The survey first introduces LLMs' key features and common training schemes, then delves into their applications in modular AD pipelines and end-to-end AD, respectively, followed by discussions on open challenges and future directions. Through this in-depth analysis, we aim to provide insights and inspiration for researchers and practitioners working at the intersection of AI and autonomous vehicles, ultimately contributing to safer, smarter, and more human-centric AD technologies.
翻译:大型语言模型(LLMs)作为基于海量文本语料训练、具有卓越语言理解与生成能力的人工智能模型,正在变革自动驾驶(AD)领域。随着自动驾驶系统从基于规则和优化的方法演进至基于学习的技术(如深度强化学习),当前正迈向第三个更先进的阶段:由LLMs赋能的基于知识的自动驾驶。这一转变有望推动自动驾驶向类人化方向发展。然而,将LLMs整合到自动驾驶系统中仍面临实时推理、安全保证与部署成本等方面的挑战。本综述对利用LLMs推进自动驾驶的最新进展进行了全面而批判性的回顾,重点关注其在模块化自动驾驶流程与端到端自动驾驶系统中的应用。我们重点梳理了关键进展,指出了紧迫挑战,并提出了具有前景的研究方向,以弥合LLMs与自动驾驶之间的鸿沟,从而促进更具类人特性的自动驾驶系统的发展。本文首先介绍了LLMs的核心特性与常见训练范式,继而分别深入探讨其在模块化自动驾驶流程和端到端自动驾驶中的具体应用,随后讨论了当前面临的开放挑战与未来发展方向。通过这一深入分析,我们旨在为人工智能与自动驾驶交叉领域的研究者与实践者提供见解与启发,最终为开发更安全、更智能、更以人为中心的自动驾驶技术贡献力量。