Modern deep neural networks heavily rely on a large number of multiply-accumulate operations, which constitute the predominant computational cost. To address this, Look-Up Table (LUT)-based matrix multiplications have emerged as a promising alternative for reducing the computational cost and time of the multiply-accumulate operations in a neural network. However, the LUT-based neural network still faces the scalability challenge due to the inherent limitations of LUT-based matrix multiplication. To mitigate these scalability limitations, this paper proposes a scalable and energy-efficient LUT-based approximate matrix multiplication unit (LUT-MU) constituting the basic component of the neural networks by integrating a pruning strategy on the MADDNESS algorithm, a LUT-based matrix multiplication methodology. With increasing problem size and precision demands in matrix multiplication, our proposed LUT-MU architecture effectively constrains resource expansion. The case study shows that deploying our LUT-MU in neural network architectures, including fully connected layers (MNIST) and ResNets (CIFAR-10, ImageNet)-on XCZU7EV and XCZU19EG FPGAs, produces up to $1.6 \times$ throughput improvement and $4.2 \times$ energy efficiency gains over mainstream CUDA-based network implementations, and $1.8\times$ energy efficiency compared to leading quantised neural network implementations, with moderate impact on accuracy. Compared to original MADDNESS-based neural networks, our LUT-MU shows $1.3$ to $2.6\times$ resource savings based on various resolution configuration settings of MADDNESS.

翻译：现代深度神经网络严重依赖大量乘加运算，这是主要的计算成本。为解决这一问题，基于查找表（LUT）的矩阵乘法被提出，作为降低神经网络中乘加运算计算成本与时间的一种有前景的替代方案。然而，由于基于LUT的矩阵乘法固有的局限性，基于LUT的神经网络仍面临可扩展性挑战。为缓解这些可扩展性限制，本文提出了一种可扩展且高能效的基于LUT的近似矩阵乘法单元（LUT-MU），该单元通过将剪枝策略集成到MADDNESS算法（一种基于LUT的矩阵乘法方法）中，构成神经网络的基本组件。随着矩阵乘法问题规模与精度需求的增加，我们提出的LUT-MU架构有效限制了资源扩张。案例研究表明，将我们的LUT-MU部署于包括全连接层（MNIST）和ResNets（CIFAR-10、ImageNet）在内的神经网络架构中（基于XCZU7EV和XCZU19EG FPGA），相比于主流基于CUDA的网络实现，可实现高达1.6倍的吞吐量提升和4.2倍的能效增益；与领先的量化神经网络实现相比，能效提升1.8倍，且对精度影响适中。与基于原始MADDNESS的神经网络相比，根据MADDNESS的不同分辨率配置设置，我们的LUT-MU可实现1.3至2.6倍的资源节省。