Uncertainty estimation in Neural Networks (NNs) is vital in improving reliability and confidence in predictions, particularly in safety-critical applications. Bayesian Neural Networks (BayNNs) with Dropout as an approximation offer a systematic approach to quantifying uncertainty, but they inherently suffer from high hardware overhead in terms of power, memory, and computation. Thus, the applicability of BayNNs to edge devices with limited resources or to high-performance applications is challenging. Some of the inherent costs of BayNNs can be reduced by accelerating them in hardware on a Computation-In-Memory (CIM) architecture with spintronic memories and binarizing their parameters. However, numerous stochastic units are required to implement conventional dropout-based BayNN. In this paper, we propose the Scale Dropout, a novel regularization technique for Binary Neural Networks (BNNs), and Monte Carlo-Scale Dropout (MC-Scale Dropout)-based BayNNs for efficient uncertainty estimation. Our approach requires only one stochastic unit for the entire model, irrespective of the model size, leading to a highly scalable Bayesian NN. Furthermore, we introduce a novel Spintronic memory-based CIM architecture for the proposed BayNN that achieves more than $100\times$ energy savings compared to the state-of-the-art. We validated our method to show up to a $1\%$ improvement in predictive performance and superior uncertainty estimates compared to related works.

翻译：神经网络中的不确定性估计对于提升预测的可靠性和置信度至关重要，尤其是在安全关键型应用中。采用丢弃法作为近似的贝叶斯神经网络提供了一种量化不确定性的系统方法，但其在功耗、内存和计算方面固有地存在高硬件开销。因此，贝叶斯神经网络在资源受限的边缘设备或高性能应用中的适用性面临挑战。通过使用自旋电子存储器并在存内计算架构上对贝叶斯神经网络进行硬件加速，同时对其参数进行二值化，可以降低其部分固有成本。然而，实施基于传统丢弃法的贝叶斯神经网络需要大量随机单元。本文提出了尺度丢弃——一种针对二值神经网络的新型正则化技术，以及基于蒙特卡洛-尺度丢弃的贝叶斯神经网络，用于高效的不确定性估计。我们的方法无论模型大小如何，仅需一个随机单元即可应用于整个模型，从而实现高度可扩展的贝叶斯神经网络。此外，我们为所提出的贝叶斯神经网络引入了一种新型基于自旋电子存储器的存内计算架构，与现有技术相比，实现了超过100倍的能耗节省。我们验证了该方法在与相关工作的对比中，预测性能提升高达1%，且不确定性估计更优。