The majority of the research on the quantization of Deep Neural Networks (DNNs) is focused on reducing the precision of tensors visible by high-level frameworks (e.g., weights, activations, and gradients). However, current hardware still relies on high-accuracy core operations. Most significant is the operation of accumulating products. This high-precision accumulation operation is gradually becoming the main computational bottleneck. This is because, so far, the usage of low-precision accumulators led to a significant degradation in performance. In this work, we present a simple method to train and fine-tune high-end DNNs, to allow, for the first time, utilization of cheaper, $12$-bits accumulators, with no significant degradation in accuracy. Lastly, we show that as we decrease the accumulation precision further, using fine-grained gradient approximations can improve the DNN accuracy.

翻译：关于深度神经网络（DNN）量化的研究大多聚焦于降低高层框架可见张量（如权重、激活值和梯度）的精度。然而，当前硬件仍依赖高精度核心运算，其中最为重要的是乘积累加操作。这种高精度累加操作正逐渐成为主要计算瓶颈——因为迄今使用低精度累加器会导致显著的性能退化。本文提出一种简单方法，首次实现对高端DNN进行训练与微调，使其能够利用成本更低的12位累加器，且精度无明显下降。进一步研究表明，当降低累加精度时，采用细粒度梯度近似可提升DNN精度。