Efficient inference of Deep Neural Networks (DNNs) is essential to making AI ubiquitous. Two important algorithmic techniques have shown promise for enabling efficient inference - sparsity and binarization. These techniques translate into weight sparsity and weight repetition at the hardware-software level enabling the deployment of DNNs with critically low power and latency requirements. We propose a new method called signed-binary networks to improve efficiency further (by exploiting both weight sparsity and weight repetition together) while maintaining similar accuracy. Our method achieves comparable accuracy on ImageNet and CIFAR10 datasets with binary and can lead to 69% sparsity. We observe real speedup when deploying these models on general-purpose devices and show that this high percentage of unstructured sparsity can lead to a further reduction in energy consumption on ASICs.

翻译：深度神经网络（DNNs）的高效推理对于实现人工智能无处不在至关重要。两种重要的算法技术——稀疏化和二值化——在实现高效推理方面展现出巨大潜力。这些技术在软硬件层面转化为权重稀疏性和权重重复性，使得部署具有极低功耗和延迟要求的DNNs成为可能。我们提出了一种名为带符号二值网络的新方法，通过同时利用权重稀疏性和权重重复性进一步提升效率，同时保持相近的精度。该方法在ImageNet和CIFAR10数据集上达到了与二值网络相当的精度，并可实现69%的稀疏度。我们观察到在通用设备上部署这些模型时实现了实际加速，并表明这种高比例非结构化稀疏度可进一步降低ASICs上的能耗。