Object detectors deployed in safety-critical environments can fail silently, e.g. missing pedestrians, workers, or other safety-critical objects without emitting any warning. Traditional Out Of Distribution (OOD) detection methods focus on identifying unfamiliar inputs, but do not directly predict functional failures of the detector itself. We introduce Knowledge Guided Failure Prediction (KGFP), a representation-based monitoring framework that treats missed safety-critical detections as anomalies to be detected at runtime. KGFP measures semantic misalignment between internal object detector features and visual foundation model embeddings using a dual-encoder architecture with an angular distance metric. A key property is that when either the detector is operating outside its competence or the visual foundation model itself encounters novel inputs, the two embeddings diverge, producing a high-angle signal that reliably flags unsafe images. We compare our novel KGFS method to baseline OOD detection methods. On COCO person detection, applying KGFP as a selective-prediction gate raises person recall among accepted images from 64.3% to 84.5% at 5% False Positive Rate (FPR), and maintains strong performance across six COCO-O visual domains, outperforming OOD baselines by large margins. Our code, models, and features are published at https://gitlab.cc-asp.fraunhofer.de/iosb_public/KGFP.

翻译：部署在安全关键环境中的目标检测器可能无声无息地失效，例如遗漏行人、工人或其他安全关键目标而不发出任何警告。传统的分布外（OOD）检测方法专注于识别不熟悉的输入，但并未直接预测检测器本身的功能失效。我们提出知识引导的失效预测（KGFP），这是一种基于表征的监控框架，将遗漏安全关键检测视为需要在运行时检测的异常。KGFP通过采用带有角距离度量的双编码器架构，测量内部目标检测器特征与视觉基础模型嵌入之间的语义错位。一个关键特性是，当检测器在其能力范围之外运行或视觉基础模型本身遇到新颖输入时，两个嵌入会发散，产生高角度信号，从而可靠地标记不安全图像。我们将新颖的KGFP方法与基线OOD检测方法进行比较。在COCO人物检测上，将KGFP作为选择性预测门控应用，在5%假阳性率（FPR）下，将接受图像中的人物召回率从64.3%提高到84.5%，并在六个COCO-O视觉域上保持强劲性能，大幅优于OOD基线方法。我们的代码、模型和特征发布在https://gitlab.cc-asp.fraunhofer.de/iosb_public/KGFP。