Deep Neural Networks are increasingly adopted in critical tasks that require a high level of safety, e.g., autonomous driving. While state-of-the-art verifiers can be employed to check whether a DNN is unsafe w.r.t. some given property (i.e., whether there is at least one unsafe input configuration), their yes/no output is not informative enough for other purposes, such as shielding, model selection, or training improvements. In this paper, we introduce the #DNN-Verification problem, which involves counting the number of input configurations of a DNN that result in a violation of a particular safety property. We analyze the complexity of this problem and propose a novel approach that returns the exact count of violations. Due to the #P-completeness of the problem, we also propose a randomized, approximate method that provides a provable probabilistic bound of the correct count while significantly reducing computational requirements. We present experimental results on a set of safety-critical benchmarks that demonstrate the effectiveness of our approximate method and evaluate the tightness of the bound.

翻译：深度神经网络日益广泛应用于需要高水平安全性的关键任务中，例如自动驾驶。虽然最先进的验证工具可用于检查深度神经网络是否违反某些给定属性（即是否存在至少一个不安全输入配置），但其二元输出对于其他目的（如屏蔽、模型选择或训练改进）信息量不足。本文提出#DNN验证问题，该问题涉及统计导致特定安全属性被违反的深度神经网络输入配置的数量。我们分析了该问题的复杂性，并提出了一种返回精确违反次数的新方法。鉴于该问题的#P完全性，我们还提出了一种随机近似方法，该方法能在显著降低计算开销的同时，提供正确计数的可证明概率界限。我们在一组安全关键基准测试上进行了实验，结果证明了我们近似方法的有效性，并评估了界限的紧致性。