The biggest challenge for the deployment of Deep Neural Networks (DNNs) close to the generated data on edge devices is their size, i.e., memory footprint and computational complexity. Both are significantly reduced with quantization. With the resulting lower word-length, the energy efficiency of DNNs increases proportionally. However, lower word-length typically causes accuracy degradation. To counteract this effect, the quantized DNN is retrained. Unfortunately, training costs up to 5000x more energy than the inference of the quantized DNN. To address this issue, we propose a post-training quantization flow without the need for retraining. For this, we investigated different quantization options. Furthermore, our analysis systematically assesses the impact of reduced word-lengths of weights and activations revealing a clear trend for the choice of word-length. Both aspects have not been systematically investigated so far. Our results are independent of the depth of the DNNs and apply to uniform quantization, allowing fast quantization of a given pre-trained DNN. We excel state-of-the-art for 6 bit by 2.2% Top-1 accuracy for ImageNet. Without retraining, our quantization to 8 bit surpasses floating-point accuracy.

翻译：在边缘设备上部署接近生成数据的深神经网络的最大挑战在于其大小,即内存足迹和计算复杂度。 两者都随着量化而大大降低。 由此导致的单长较低, DNN的能源效率成比例地提高。 但是, 低单长通常导致精度降解。 为了抵消这一效应, 量化的 DNN 重新培训。 不幸的是, 培训费用比量化的 DNN 的推论高出5000x的能量。 为了解决这个问题, 我们提出无需再培训的训练后量化流程。 为此, 我们调查了不同的量化选项。 此外, 我们的分析系统地评估了减字长的重量的影响, 并激活了对字长选择的明确趋势。 目前尚未对这两个方面进行系统调查。 我们的结果独立于 DNN 的深度, 并适用于统一的量化, 从而可以快速量化一个事先经过培训的 DNNN。 我们的状态, 6比2.2% 最高一级精确度高出6个百分点, 至最高一级图像网络。