Neural Network designs are quite diverse, from VGG-style to ResNet-style, and from Convolutional Neural Networks to Transformers. Towards the design of efficient accelerators, many works have adopted a dataflow-based, inter-layer pipelined architecture, with a customised hardware towards each layer, achieving ultra high throughput and low latency. The deployment of neural networks to such dataflow architecture accelerators is usually hindered by the available on-chip memory as it is desirable to preload the weights of neural networks on-chip to maximise the system performance. To address this, networks are usually compressed before the deployment through methods such as pruning, quantization and tensor decomposition. In this paper, a framework for mapping CNNs onto FPGAs based on a novel tensor decomposition method called Mixed-TD is proposed. The proposed method applies layer-specific Singular Value Decomposition (SVD) and Canonical Polyadic Decomposition (CPD) in a mixed manner, achieving 1.73x to 10.29x throughput per DSP to state-of-the-art CNNs. Our work is open-sourced: https://github.com/Yu-Zhewen/Mixed-TD

翻译：神经网络设计风格多样，从VGG结构到ResNet结构，从卷积神经网络到Transformer。为实现高效加速器，许多研究采用基于数据流的层间流水线架构，针对每一层定制硬件，从而实现超高吞吐和低延迟。神经网络部署到此类数据流架构加速器时，通常受片上存储资源限制，因为理想情况下需将网络权重预加载至片上以最大化系统性能。为解决这一问题，网络在部署前通常通过剪枝、量化和张量分解等方法进行压缩。本文提出一种基于新型张量分解方法Mixed-TD的CNN到FPGA映射框架。该方法以混合方式应用逐层奇异值分解（SVD）和规范多元分解（CPD），在先进CNN上实现了每DSP 1.73倍至10.29倍的吞吐提升。本工作已开源：https://github.com/Yu-Zhewen/Mixed-TD