Uncertainty estimation in Neural Networks (NNs) is vital in improving reliability and confidence in predictions, particularly in safety-critical applications. Bayesian Neural Networks (BayNNs) with Dropout as an approximation offer a systematic approach to quantifying uncertainty, but they inherently suffer from high hardware overhead in terms of power, memory, and computation. Thus, the applicability of BayNNs to edge devices with limited resources or to high-performance applications is challenging. Some of the inherent costs of BayNNs can be reduced by accelerating them in hardware on a Computation-In-Memory (CIM) architecture with spintronic memories and binarizing their parameters. However, numerous stochastic units are required to implement conventional dropout-based BayNN. In this paper, we propose the Scale Dropout, a novel regularization technique for Binary Neural Networks (BNNs), and Monte Carlo-Scale Dropout (MC-Scale Dropout)-based BayNNs for efficient uncertainty estimation. Our approach requires only one stochastic unit for the entire model, irrespective of the model size, leading to a highly scalable Bayesian NN. Furthermore, we introduce a novel Spintronic memory-based CIM architecture for the proposed BayNN that achieves more than $100\times$ energy savings compared to the state-of-the-art. We validated our method to show up to a $1\%$ improvement in predictive performance and superior uncertainty estimates compared to related works.

翻译：神经网络中的不确定性估计对于提高预测的可靠性和置信度至关重要，尤其是在安全关键型应用中。基于Dropout近似的贝叶斯神经网络提供了一种量化不确定性的系统方法，但其在功耗、内存和计算方面固有地存在较高的硬件开销。因此，贝叶斯神经网络在资源受限的边缘设备或高性能应用中的适用性具有挑战性。通过在基于自旋电子存储器存内计算架构上进行硬件加速并对其参数进行二值化，可以降低贝叶斯神经网络的固有成本。然而，实现传统的基于Dropout的贝叶斯神经网络需要大量随机单元。本文提出了一种名为Scale Dropout的新型二值神经网络正则化技术，以及基于蒙特卡洛-Scale Dropout的贝叶斯神经网络，用于高效的不确定性估计。我们的方法无论模型规模大小，仅需一个随机单元即可应用于整个模型，从而构建出高度可扩展的贝叶斯神经网络。此外，我们还为所提出的贝叶斯神经网络引入了一种基于自旋电子存储器的新型存内计算架构，与现有最先进技术相比，实现了超过100倍的能耗节省。实验验证表明，与相关工作相比，我们的方法在预测性能上提升高达1%，并具有更优的不确定性估计。