Efficiently deploying deep neural networks on low-resource edge devices is challenging due to their ever-increasing resource requirements. To address this issue, researchers have proposed multiplication-free neural networks, such as Power-of-Two quantization, or also known as Shift networks, which aim to reduce memory usage and simplify computation. However, existing low-bit Shift networks are not as accurate as their full-precision counterparts, typically suffering from limited weight range encoding schemes and quantization loss. In this paper, we propose the DenseShift network, which significantly improves the accuracy of Shift networks, achieving competitive performance to full-precision networks for vision and speech applications. In addition, we introduce a method to deploy an efficient DenseShift network using non-quantized floating-point activations, while obtaining 1.6X speed-up over existing methods. To achieve this, we demonstrate that zero-weight values in low-bit Shift networks do not contribute to model capacity and negatively impact inference computation. To address this issue, we propose a zero-free shifting mechanism that simplifies inference and increases model capacity. We further propose a sign-scale decomposition design to enhance training efficiency and a low-variance random initialization strategy to improve the model's transfer learning performance. Our extensive experiments on various computer vision and speech tasks demonstrate that DenseShift outperforms existing low-bit multiplication-free networks and achieves competitive performance compared to full-precision networks. Furthermore, our proposed approach exhibits strong transfer learning performance without a drop in accuracy. Our code was released on GitHub.

翻译：在低资源边缘设备上高效部署深度神经网络因其不断增长的资源需求而面临挑战。为解决此问题，研究者提出了免乘法神经网络，例如二次幂量化（亦称移位网络），旨在减少内存使用并简化计算。然而，现有低比特移位网络的精度不及全精度对应模型，通常受限于权重范围编码方案有限及量化损失。本文提出DenseShift网络，显著提升了移位网络的精度，在视觉与语音应用中达到与全精度网络相媲美的性能。此外，我们引入一种方法，通过使用非量化浮点激活值部署高效DenseShift网络，同时相较现有方法获得1.6倍加速。为此，我们证明低比特移位网络中的零权重值不贡献模型容量，并对推理计算产生负面影响。针对此问题，我们提出零自由移位机制以简化推理并提高模型容量。进一步提出符号-尺度分解设计以提升训练效率，以及低方差随机初始化策略以改善模型的迁移学习性能。我们在多种计算机视觉与语音任务上的广泛实验表明，DenseShift优于现有低比特免乘法网络，并与全精度网络性能相当。此外，我们所提方法展现出强大的迁移学习能力且准确率未下降。代码已在GitHub上开源。