Power-of-two (PoT) quantization significantly reduces the size of deep neural networks (DNNs) and replaces multiplications with bit-shift operations for inference. Prior work has shown that PoT-quantized DNNs can preserve accuracy for tasks such as image classification; however, their performance on resource-constrained edge devices remains insufficiently understood. While general-purpose edge CPUs and GPUs do not provide optimized backends for bit-shift operations, custom hardware accelerators can better exploit PoT quantization by implementing dedicated shift-based processing elements. However, deploying PoT-quantized models on such accelerators is challenging due to limited support in existing inference frameworks. In addition, the impact of different PoT quantization strategies on hardware design, performance, and energy efficiency during full inference has not been systematically explored. To address these challenges, we propose PoTAcc, an open-source end-to-end pipeline for accelerating and evaluating PoT-quantized DNNs on resource-constrained edge devices. PoTAcc enables seamless preparation and deployment of PoT-quantized models via TensorFlow Lite (TFLite) across heterogeneous platforms, including CPU-only systems and hybrid CPU-FPGA systems with custom accelerators. We design shift-based processing element (shift-PE) accelerators for three PoT quantization methods and implement them on two FPGA platforms. We evaluate accuracy, performance, energy efficiency, and resource utilization across a range of models, including CNNs and Transformer-based architectures. Results show that our CPU-accelerator design achieves up to 3.6x speedup and 78% energy reduction compared to CPU-only execution for PoT-quantized DNNs on PYNQ-Z2 and Kria boards. The code will be publicly released at https://github.com/gicLAB/PoTAcc

翻译：2的幂次量化通过比特移位操作替代乘法运算，显著缩减了深度神经网络的规模并简化其推理过程。已有研究表明，2的幂次量化后的深度神经网络在图像分类等任务中能保持准确率；然而，其在资源受限的边缘设备上的性能表现尚未得到充分认知。通用型边缘CPU和GPU并未为比特移位运算提供优化的后端支持，而定制化硬件加速器可通过实现专用移位处理单元来更好地利用2的幂次量化特性。但受限于现有推理框架的有限支持，在此类加速器上部署2的幂次量化模型仍面临挑战。此外，不同2的幂次量化策略对硬件设计、性能及完整推理过程中能效的影响尚未得到系统探究。为应对上述挑战，我们提出PoTAcc——一个用于在资源受限边缘设备上加速和评估2的幂次量化深度神经网络的开源端到端流水线。PoTAcc支持通过TensorFlow Lite在异构平台（包括纯CPU系统及配备定制加速器的CPU-FPGA混合系统）上无缝完成2的幂次量化模型的准备与部署。我们针对三种2的幂次量化方法设计了基于移位处理单元的加速器（shift-PE），并在两种FPGA平台上实现。通过对涵盖CNN和基于Transformer架构的多种模型进行精度、性能、能效及资源利用率的评估，结果显示：在PYNQ-Z2和Kria开发板上，相较于纯CPU执行，我们的CPU-加速器协同设计可实现最高3.6倍的加速比和78%的能耗降低。相关代码将在https://github.com/gicLAB/PoTAcc 公开发布。