Large Language Models (LLMs) are increasingly used for decision-making and planning in autonomous driving, showing promising reasoning capabilities and potential to generalize across diverse traffic situations. However, current LLM-based driving agents lack explicit mechanisms to enforce traffic rules and often struggle to reliably detect small, safety-critical objects such as traffic lights and signs. To address this limitation, we introduce TLS-Assist, a modular redundancy layer that augments LLM-based autonomous driving agents with explicit traffic light and sign recognition. TLS-Assist converts detections into structured natural language messages that are injected into the LLM input, enforcing explicit attention to safety-critical cues. The framework is plug-and-play, model-agnostic, and supports both single-view and multi-view camera setups. We evaluate TLS-Assist in a closed-loop setup on the LangAuto benchmark in CARLA. The results demonstrate relative driving performance improvements of up to 14% over LMDrive and 7% over BEVDriver, while consistently reducing traffic light and sign infractions. We publicly release the code and models on https://github.com/iis-esslingen/TLS-Assist.

翻译：大型语言模型（LLMs）在自动驾驶的决策与规划中应用日益广泛，展现出优异的推理能力及适应多样化交通场景的潜力。然而，当前基于LLM的驾驶智能体缺乏强制遵守交通规则的显式机制，且难以可靠地检测交通信号灯、标志等对安全至关重要的微小目标。为克服此局限，我们提出TLS-Assist——一种模块化冗余层，通过显式的交通信号灯与标志识别功能增强基于LLM的自动驾驶智能体。TLS-Assist将检测结果转化为结构化自然语言信息并注入LLM输入，强制模型关注安全关键线索。该框架具备即插即用、模型无关的特性，支持单目与多目相机配置。我们在CARLA仿真环境的LangAuto基准测试中采用闭环设置评估TLS-Assist。实验结果表明，相较于LMDrive和BEVDriver，其驾驶性能分别提升最高达14%和7%，同时持续降低交通信号灯与标志违规行为。代码与模型已公开于https://github.com/iis-esslingen/TLS-Assist。