As Deep Neural Networks (DNNs) are increasingly deployed in safety critical and privacy sensitive applications such as autonomous driving and biometric authentication, it is critical to understand the fault-tolerance nature of DNNs. Prior work primarily focuses on metrics such as Failures In Time (FIT) rate and the Silent Data Corruption (SDC) rate, which quantify how often a device fails. Instead, this paper focuses on quantifying the DNN accuracy given that a transient error has occurred, which tells us how well a network behaves when a transient error occurs. We call this metric Resiliency Accuracy (RA). We show that existing RA formulation is fundamentally inaccurate, because it incorrectly assumes that software variables (model weights/activations) have equal faulty probability under hardware transient faults. We present an algorithm that captures the faulty probabilities of DNN variables under transient faults and, thus, provides correct RA estimations validated by hardware. To accelerate RA estimation, we reformulate RA calculation as a Monte Carlo integration problem, and solve it using importance sampling driven by DNN specific heuristics. Using our lightweight RA estimation method, we show that transient faults lead to far greater accuracy degradation than what todays DNN resiliency tools estimate. We show how our RA estimation tool can help design more resilient DNNs by integrating it with a Network Architecture Search framework.

翻译：随着深度神经网络（DNN）在自动驾驶和生物特征认证等安全关键与隐私敏感型应用中的广泛部署，理解DNN的容错特性变得至关重要。现有工作主要关注失效时间（FIT）率和静默数据损坏（SDC）率等指标，这些指标量化了设备发生故障的频率。相反，本文聚焦于在瞬态错误发生后量化DNN精度，揭示网络在遇到瞬态错误时的行为表现。我们将此度量指标称为弹性精度（RA）。研究表明，现有RA表述存在根本性不准确——它错误地假设软件变量（模型权重/激活值）在硬件瞬态故障下具有相同的出错概率。本文提出一种算法，可捕获瞬态故障下DNN变量的出错概率，从而提供经硬件验证的正确RA估计。为加速RA估计，我们将RA计算重构为蒙特卡洛积分问题，并利用由DNN特定启发式规则驱动的重采样方法进行求解。通过轻量级RA估计方法，我们发现瞬态故障导致的精度下降远超当前DNN弹性工具的评估结果。最后展示如何将RA估计工具与网络架构搜索框架集成，以设计更具弹性的DNN。