Real-world autonomous driving must adhere to complex human social rules that extend beyond legally codified traffic regulations. Many of these semantic constraints, such as yielding to emergency vehicles, complying with traffic officers' gestures, or stopping for school buses, are intuitive for humans yet difficult to encode explicitly. Although large vision-language models (VLMs) can interpret such semantics, their inference cost makes them impractical for real-time deployment. This work proposes LSRE, a Latent Semantic Rule Encoding framework that converts sparsely sampled VLM judgments into decision boundaries within the latent space of a recurrent world model. By encoding language-defined safety semantics into a lightweight latent classifier, LSRE enables real-time semantic risk assessment at 10 Hz without per-frame VLM queries. Experiments on six semantic-failure scenarios in CARLA demonstrate that LSRE attains semantic risk detection accuracy comparable to a large VLM baseline, while providing substantially earlier hazard anticipation and maintaining low computational latency. LSRE further generalizes to rarely seen semantic-similar test cases, indicating that language-guided latent classification offers an effective and deployable mechanism for semantic safety monitoring in autonomous driving.

翻译：现实世界的自动驾驶必须遵循复杂的人类社会规则，这些规则超出了法律明文规定的交通法规。许多此类语义约束，例如为紧急车辆让行、遵守交通警察的手势指示或为校车停车，对人类而言是直观的，但难以进行显式编码。尽管大型视觉语言模型能够解释此类语义，但其推理成本使其难以用于实时部署。本文提出LSRE，一种潜在语义规则编码框架，它将稀疏采样的大型视觉语言模型判断转换为循环世界模型潜在空间中的决策边界。通过将语言定义的安全语义编码到一个轻量级的潜在分类器中，LSRE能够在无需逐帧查询大型视觉语言模型的情况下，以10 Hz的频率实现实时语义风险评估。在CARLA仿真环境中对六种语义失效场景的实验表明，LSRE获得的语义风险检测准确率与大型视觉语言模型基线相当，同时能显著更早地预测危险，并保持较低的计算延迟。LSRE还能泛化到极少见的语义相似测试案例，这表明语言引导的潜在分类为自动驾驶的语义安全监控提供了一种有效且可部署的机制。