There has been increasing interest in characterising the error behaviour of systems which contain deep learning models before deploying them into any safety-critical scenario. However, characterising such behaviour usually requires large-scale testing of the model that can be extremely computationally expensive for complex real-world tasks. For example, tasks involving compute intensive object detectors as one of their components. In this work, we propose an approach that enables efficient large-scale testing using simplified low-fidelity simulators and without the computational cost of executing expensive deep learning models. Our approach relies on designing an efficient surrogate model corresponding to the compute intensive components of the task under test. We demonstrate the efficacy of our methodology by evaluating the performance of an autonomous driving task in the Carla simulator with reduced computational expense by training efficient surrogate models for PIXOR and CenterPoint LiDAR detectors, whilst demonstrating that the accuracy of the simulation is maintained.

翻译：人们越来越关注含有深层学习模型的系统的错误行为,这些系统的错误行为在将其应用到任何安全关键情景之前,其特征通常要求对模型进行大规模测试,而这种模型对于复杂的现实世界任务来说,在计算成本方面成本极高。例如,将密集天体探测器作为其组成部分之一进行计算的任务。在这项工作中,我们建议采用一种方法,利用简化的低纤维模拟器进行高效的大规模测试,而不使用昂贵的深层学习模型的计算成本。我们的方法依赖于设计一个与测试中任务计算密集部分相对应的有效替代模型。我们通过对PIXOR和Centpoint LiDAR探测器的有效代用模型进行培训,对卡拉模拟器自动驾驶任务进行绩效评估,降低计算成本,从而证明我们的方法的有效性,同时证明模拟的准确性。