Vision-language models (VLMs) are increasingly used for scene understanding in autonomous driving, but robustness analysis often relies on task-agnostic embedding stability alone. We study whether corruption-induced embedding drift predicts changes in a task-aligned hazard score derived from CLIP image-text similarities. Using controlled corruptions on BDD100K road scenes, we compare embedding drift against margin drift, defined as the change in hazard score under perturbation. The relationship is highly corruption-dependent: some families exhibit strong coupling between representation drift and decision drift, while others induce hazardous decision instability despite relatively modest embedding change. Furthermore, corruption families differ in failure direction: most suppress hazard detections via false negatives, while occlusion instead triggers false alarms, suggesting that benchmark design should account for asymmetric failure modes, not just overall instability rates. These results suggest that robustness benchmarks should include task-aligned stability measures in addition to embedding-level perturbation statistics.

翻译：视觉-语言模型（VLM）日益广泛地应用于自动驾驶场景理解，但其鲁棒性分析通常仅依赖于任务无关的嵌入稳定性。本研究探究了由数据损坏引发的嵌入漂移是否能预测基于CLIP图像-文本相似性计算的任务对齐危险分数的变化。通过在BDD100K道路场景中施加受控扰动，我们比较了嵌入漂移与边界漂移（即扰动下危险分数的变化）。两者关系高度依赖于扰动的类型：部分扰动族表现出表示漂移与决策漂移之间的强耦合关系，而其他扰动族则在嵌入变化相对较小的情况下仍会引发危险的决策不稳定性。此外，不同的扰动族在失效方向上存在差异：多数扰动通过假阴性抑制危险检测，而遮挡则反而引发虚警。这表明基准测试设计应关注非对称故障模式，而非仅关注总体不稳定率。上述结果表明，鲁棒性基准测试除嵌入层扰动统计指标外，还应纳入任务对齐的稳定性度量指标。