Convolutional Neural Networks (CNNs) are used in a wide range of applications, with full-precision CNNs achieving high accuracy at the expense of portability. Recent progress in quantization techniques has demonstrated that sub-byte Quantized Neural Networks (QNNs) achieve comparable or superior accuracy while significantly reducing the computational cost and memory footprint. However, sub-byte computation on commodity hardware is sub-optimal due to the lack of support for such precision. In this paper, we introduce Sparq, a Sub-byte vector Processor designed for the AcceleRation of QNN inference. This processor is based on a modified version of Ara, an open-source 64-bit RISC-V ``V'' compliant processor. Sparq is implemented in GLOBAL FOUNDRIES 22FDX FD-SOI technology and extends the Instruction Set Architecture (ISA) by adding a new multiply-shift-accumulate instruction to improve sub-byte computation effciency. The floating-point unit is also removed to minimize area and power usage. To demonstrate Sparq performance, we implement an ultra-low-precision (1-bit to 4-bit) vectorized conv2d operation taking advantage of the dedicated hardware. We show that Sparq can significantly accelerate sub-byte computations with respectively 3.2 times, and 1.7 times acceleration over an optimized 16-bit 2D convolution for 2-bit and 4-bit quantization.

翻译：摘要：卷积神经网络（CNN）广泛应用于各类场景，全精度CNN虽能实现高准确率，却以牺牲可移植性为代价。量化技术的最新进展表明，亚字节量化神经网络（QNN）在显著降低计算开销和内存占用的同时，可实现与全精度网络相当甚至更优的准确率。然而，由于缺乏对亚字节精度的硬件支持，当前通用硬件在执行亚字节计算时性能欠佳。本文提出Sparq——一款面向QNN推理加速的亚字节向量处理器。该处理器基于开源64位RISC-V向量指令集兼容处理器Ara的改进版本实现，采用格芯22FDX FD-SOI工艺制造。通过新增乘-移位-累加指令扩展指令集架构（ISA），有效提升了亚字节计算效率。为最小化面积与功耗，处理器还移除了浮点运算单元。为验证Sparq性能，我们利用专用硬件实现了超低精度（1比特至4比特）向量化二维卷积运算。实验表明，相较于优化后的16位二维卷积，Sparq在2比特与4比特量化场景下分别实现了3.2倍和1.7倍的亚字节计算加速。