The modern pervasiveness of large-scale deep neural networks (NNs) is driven by their extraordinary performance on complex problems but is also plagued by their sudden, unexpected, and often catastrophic failures, particularly on challenging scenarios. Existing algorithms that provide risk-awareness to NNs are complex and ad-hoc. Specifically, these methods require significant engineering changes, are often developed only for particular settings, and are not easily composable. Here we present capsa, a framework for extending models with risk-awareness. Capsa provides a methodology for quantifying multiple forms of risk and composing different algorithms together to quantify different risk metrics in parallel. We validate capsa by implementing state-of-the-art uncertainty estimation algorithms within the capsa framework and benchmarking them on complex perception datasets. We demonstrate capsa's ability to easily compose aleatoric uncertainty, epistemic uncertainty, and bias estimation together in a single procedure, and show how this approach provides a comprehensive awareness of NN risk.

翻译：现代大规模深度神经网络（NN）的普遍应用源于其在复杂问题上的卓越性能，但也因其在挑战性场景中出现的突发、意外且往往是灾难性的失败而备受困扰。现有的为神经网络提供风险感知能力的算法复杂且具有临时性。具体而言，这些方法需要显著的工程改动，通常仅为特定场景开发，且难以组合。本文提出capsa，一个用于扩展模型风险感知能力的框架。Capsa提供了一种量化多种风险形式的方法，并可将不同算法组合以同时量化不同的风险指标。我们通过在capsa框架内实现最先进的不确定性估计算法，并在复杂感知数据集上进行基准测试来验证capsa的性能。我们展示了capsa能够轻松地在单一流程中组合偶然不确定性、认知不确定性和偏差估计，并说明了该方法如何提供对神经网络风险的全面感知。