Sensing and Perception (S&P) is a crucial component of an autonomous system (such as a robot), especially when deployed in highly dynamic environments where it is required to react to unexpected situations. This is particularly true in case of Autonomous Vehicles (AVs) driving on public roads. However, the current evaluation metrics for perception algorithms are typically designed to measure their accuracy per se and do not account for their impact on the decision making subsystem(s). This limitation does not help developers and third party evaluators to answer a critical question: is the performance of a perception subsystem sufficient for the decision making subsystem to make robust, safe decisions? In this paper, we propose a simulation-based methodology towards answering this question. At the same time, we show how to analyze the impact of different kinds of sensing and perception errors on the behavior of the autonomous system.

翻译：遥感和感知(S&P)是自主系统(如机器人)的关键组成部分,特别是在高度动态的环境中部署,需要它应对意外情况时尤其如此。对于在公共道路上驾驶的自动车辆,情况尤其如此。然而,目前对感知算法的评价指标通常设计为衡量其准确性,而没有考虑到其对决策子系统的影响。这一限制无助于开发者和第三方评价者回答一个关键问题:感知子系统的运行是否足以使决策子系统做出稳健、安全的决定?在本文件中,我们提出一个基于模拟的方法来回答这个问题。与此同时,我们展示了如何分析不同感知和感知错误对自主系统行为的影响。