The development of model compression is continuously motivated by the evolution of various neural network accelerators with ASIC or FPGA. On the algorithm side, the ultimate goal of quantization or pruning is accelerating the expensive DNN computations on low-power hardware. However, such a "design-and-deploy" workflow faces under-explored challenges in the current hardware-algorithm co-design community. First, although the state-of-the-art quantization algorithm can achieve low precision with negligible degradation of accuracy, the latest deep learning framework (e.g., PyTorch) can only support non-customizable 8-bit precision, data format, and parameter extraction. Secondly, the objective of quantization is to enable the computation with low-precision data. However, the current SoTA algorithm treats the quantized integer as an intermediate result, while the final output of the quantizer is the "discretized" floating-point values, ignoring the practical needs and adding additional workload to hardware designers for integer parameter extraction and layer fusion. Finally, the compression toolkits designed by the industry are constrained to their in-house product or a handful of algorithms. The limited degree of freedom in the current toolkit and the under-explored customization hinder the prototype ASIC or FPGA-based accelerator design. To resolve these challenges, we propose Torch2Chip, an open-sourced, fully customizable, and high-performance toolkit that supports user-designed compression followed by automatic model fusion and parameter extraction. Torch2Chip incorporates the hierarchical design workflow, and the user-customized compression algorithm will be directly packed into the deployment-ready format for prototype chip verification with either CNN or vision transformer (ViT). The code is available at https://github.com/SeoLabCornell/torch2chip.

翻译：模型压缩的发展持续受到基于ASIC或FPGA的各种神经网络加速器演进的驱动。在算法层面，量化或剪枝的最终目标是在低功耗硬件上加速昂贵的深度神经网络计算。然而，这种"设计并部署"的工作流程在当前硬件-算法协同设计社区中面临诸多未被充分探索的挑战。首先，尽管最先进的量化算法能够在精度可忽略的下降下实现低比特运算，但最新的深度学习框架（如PyTorch）仅支持非可定制的8位精度、数据格式和参数提取。其次，量化的目标是实现低精度数据的计算，但当前最先进的算法将量化整数视为中间结果，而量化器的最终输出是"离散化"的浮点值，这忽略了实际需求，并为硬件设计者增加了整数参数提取和层融合的额外工作负荷。最后，行业设计的压缩工具包受限于其内部产品或少数几种算法。当前工具包有限的自定义程度和未被充分探索的定制化功能阻碍了基于ASIC或FPGA的原型加速器设计。为解决这些挑战，我们提出Torch2Chip——一个开源、完全可定制且高性能的工具包，支持用户自定义压缩算法，并随后进行自动模型融合和参数提取。Torch2Chip采用分层设计工作流，用户自定义的压缩算法将直接打包为部署就绪格式，用于基于CNN或视觉Transformer（ViT）的原型芯片验证。代码已在https://github.com/SeoLabCornell/torch2chip公开。