The object perception of automated driving systems must pass quality and robustness tests before a safe deployment. Such tests typically identify true positive (TP), false-positive (FP), and false-negative (FN) detections and aggregate them to metrics. Since the literature seems to be lacking a comprehensive way to define the identification of TPs/FPs/FNs, this paper provides a checklist of relevant functional aspects and implementation details. Besides labeling policies of the test set, we cover areas of vision, occlusion handling, safety-relevant areas, matching criteria, temporal and probabilistic issues, and further aspects. Even though the checklist cannot be fully formalized, it can help practitioners minimize the ambiguity of their tests, which, in turn, makes statements on object perception more reliable and comparable.
翻译:自动驾驶系统的目标感知能力在安全部署前必须通过质量与鲁棒性测试。此类测试通常需识别真阳性(TP)、假阳性(FP)与假阴性(FN)检测结果,并将其聚合为评估指标。鉴于现有文献似乎缺乏定义TP/FP/FN识别的系统性方法,本文提出涵盖相关功能维度与实施细节的核查清单。除测试集的标注策略外,我们涉及视觉处理、遮挡处理、安全相关区域、匹配准则、时序与概率问题及其他相关方面。尽管该核查清单无法完全形式化,但可帮助从业者最大限度地减少测试的模糊性,从而提升目标感知能力评估结果的可靠性与可比性。