Neural Network designs are quite diverse, from VGG-style to ResNet-style, and from Convolutional Neural Networks to Transformers. Towards the design of efficient accelerators, many works have adopted a dataflow-based, inter-layer pipelined architecture, with a customised hardware towards each layer, achieving ultra high throughput and low latency. The deployment of neural networks to such dataflow architecture accelerators is usually hindered by the available on-chip memory as it is desirable to preload the weights of neural networks on-chip to maximise the system performance. To address this, networks are usually compressed before the deployment through methods such as pruning, quantization and tensor decomposition. In this paper, a framework for mapping CNNs onto FPGAs based on a novel tensor decomposition method called Mixed-TD is proposed. The proposed method applies layer-specific Singular Value Decomposition (SVD) and Canonical Polyadic Decomposition (CPD) in a mixed manner, achieving 1.73x to 10.29x throughput per DSP to state-of-the-art CNNs. Our work is open-sourced: https://github.com/Yu-Zhewen/Mixed-TD

翻译：神经网络设计风格多样，从VGG式到ResNet式，从卷积神经网络到Transformer。在设计高效加速器时，许多工作采用了基于数据流、层间流水线的架构，为每一层定制化硬件，实现了超高吞吐量和低延迟。神经网络部署到此类数据流架构加速器时，通常受限于片上存储容量，因为需要将网络权重预加载到片上以最大化系统性能。为解决此问题，网络通常在部署前通过剪枝、量化和张量分解等方法进行压缩。本文提出了一种基于新型张量分解方法Mixed-TD的CNN到FPGA映射框架。该方法以混合方式应用层特定奇异值分解（SVD）和规范多项式分解（CPD），在先进CNN上实现了每DSP吞吐量1.73倍至10.29倍的提升。我们的工作已开源：https://github.com/Yu-Zhewen/Mixed-TD