Many autonomous driving systems are increasingly incorporating foundation models to improve generalization and handle long-tail scenarios. However, this trend introduces two key challenges: (i) the manual and labor-intensive process of designing and integrating new models, and (ii) the lack of intelligent, dynamic scheduling mechanisms to meet strict real-time constraints. While Large Language Model (LLM)-based agents offer a promising avenue for automation, existing frameworks are ill-suited for autonomous driving. Specifically, they fail to distinguish between the fundamentally different requirements of system design and real-time scheduling, treat modules as opaque black boxes, and are not designed for continuous operation. To address these limitations, we propose DrivingAgent, a novel agent framework tailored to the dual challenges of autonomous driving system design and scheduling. In the design phase, DrivingAgent automates module development by interpreting system architecture, generating code, and validating modules via super-network training. In the scheduling phase, it employs a lightweight LLM trained with reinforcement learning to dynamically orchestrate system modules in real time, supported by a structured memory that integrates long-term storage with timestamped short-term context. Experimental results demonstrate that DrivingAgent achieves a superior speed--accuracy trade-off on both the nuScenes and Bench2Drive benchmarks.

翻译：许多自动驾驶系统正越来越多地集成基础模型，以提升泛化能力并处理长尾场景。然而，这一趋势引入了两个关键挑战：（i）设计和集成新模型的手动、劳动密集型过程，以及（ii）缺乏满足严格实时约束的智能动态调度机制。虽然基于大语言模型的智能体为自动化提供了有前景的途径，但现有框架并不适用于自动驾驶。具体而言，它们未能区分系统设计与实时调度的根本不同需求，将模块视为不透明黑盒，且并非为持续运行而设计。为应对这些局限，我们提出DrivingAgent——一种专为自动驾驶系统设计与调度双重挑战定制的新型智能体框架。在设计阶段，DrivingAgent通过解析系统架构、生成代码并利用超网络训练验证模块，自动化模块开发。在调度阶段，它采用经强化学习训练的轻量级大语言模型实时动态编排系统模块，并由整合长期存储与带时间戳短期上下文的结构化记忆提供支持。实验结果表明，DrivingAgent在nuScenes和Bench2Drive基准上均实现了更优的速度-精度权衡。