Conventional multiply-accumulate (MAC) operations have long dominated computation time for deep neural networks (DNNs). Recently, product quantization (PQ) has been successfully applied to these workloads, replacing MACs with memory lookups to pre-computed dot products. While this property makes PQ an attractive solution for model acceleration, little is understood about the associated trade-offs in terms of compute and memory footprint, and the impact on accuracy. Our empirical study investigates the impact of different PQ settings and training methods on layerwise reconstruction error and end-to-end model accuracy. When studying the efficiency of deploying PQ DNNs, we find that metrics such as FLOPs, number of parameters, and even CPU/GPU performance, can be misleading. To address this issue, and to more fairly assess PQ in terms of hardware efficiency, we design the first custom hardware accelerator to evaluate the speed and efficiency of running PQ models. We identify PQ configurations that are able to improve performance-per-area for ResNet20 by 40%-104%, even when compared to a highly optimized conventional DNN accelerator. Our hardware performance outperforms recent PQ solutions by 4x, with only a 0.6% accuracy degradation. This work demonstrates the practical and hardware-aware design of PQ models, paving the way for wider adoption of this emerging DNN approximation methodology.

翻译：传统的乘加运算（MAC）长期主导深度神经网络（DNN）的计算时间。最近，乘积量化（PQ）已成功应用于这些工作负载，用对预计算点积的存储器查找替代了MAC操作。尽管这一特性使PQ成为模型加速的有吸引力的解决方案，但关于其在计算与内存占用方面的相关权衡以及对精度的影响，人们仍知之甚少。我们的实证研究调查了不同PQ设置和训练方法对逐层重构误差及端到端模型精度的影响。在研究部署PQ深度神经网络的效率时，我们发现诸如FLOPs、参数量甚至CPU/GPU性能等指标可能具有误导性。为解决此问题并更公平地评估PQ的硬件效率，我们设计了首个定制硬件加速器，用于评估运行PQ模型的速度和效率。即使与高度优化的传统DNN加速器相比，我们识别出的PQ配置能够将ResNet20的性能密度提升40%-104%。我们的硬件性能相比近期PQ方案提升4倍，而精度仅下降0.6%。本工作展示了PQ模型的实用化与硬件感知设计，为这种新兴DNN近似方法的广泛采用铺平了道路。