The development of model compression is continuously motivated by the evolution of various neural network accelerators with ASIC or FPGA. On the algorithm side, the ultimate goal of quantization or pruning is accelerating the expensive DNN computations on low-power hardware. However, such a "design-and-deploy" workflow faces under-explored challenges in the current hardware-algorithm co-design community. First, although the state-of-the-art quantization algorithm can achieve low precision with negligible degradation of accuracy, the latest deep learning framework (e.g., PyTorch) can only support non-customizable 8-bit precision, data format, and parameter extraction. Secondly, the objective of quantization is to enable the computation with low-precision data. However, the current SoTA algorithm treats the quantized integer as an intermediate result, while the final output of the quantizer is the "discretized" floating-point values, ignoring the practical needs and adding additional workload to hardware designers for integer parameter extraction and layer fusion. Finally, the compression toolkits designed by the industry are constrained to their in-house product or a handful of algorithms. The limited degree of freedom in the current toolkit and the under-explored customization hinder the prototype ASIC or FPGA-based accelerator design. To resolve these challenges, we propose Torch2Chip, an open-sourced, fully customizable, and high-performance toolkit that supports user-designed compression followed by automatic model fusion and parameter extraction. Torch2Chip incorporates the hierarchical design workflow, and the user-customized compression algorithm will be directly packed into the deployment-ready format for prototype chip verification with either CNN or vision transformer (ViT). The code is available at https://github.com/SeoLabCornell/torch2chip.

翻译：模型压缩的发展持续得益于各类基于ASIC或FPGA的神经网络加速器的演进。在算法层面，量化或剪枝的终极目标是在低功耗硬件上加速昂贵的深度神经网络计算。然而，这种"设计-部署"流程在当前硬件-算法协同设计领域面临着尚未充分探索的挑战。首先，尽管最先进的量化算法能以可忽略的精度损失实现低比特表示，但现有深度学习框架（如PyTorch）仅支持非可定制的8位精度、数据格式及参数提取。其次，量化旨在实现低精度数据计算，但当前前沿算法将量化整数视为中间结果，而量化器的最终输出为"离散化"浮点值，这忽略了实际需求，并为硬件设计者增加了整数参数提取与层融合的额外工作量。最后，工业界设计的压缩工具包受限于其内部产品或少数几种算法，现有工具包中有限的自由度与未充分探索的可定制性阻碍了基于原型ASIC或FPGA的加速器设计。为解决上述挑战，我们提出Torch2Chip——一个开源、完全可定制、高性能的工具包，支持用户自定义压缩，并自动完成模型融合与参数提取。Torch2Chip采用分层设计工作流，用户定制的压缩算法将直接打包为可部署格式，用于基于CNN或视觉Transformer（ViT）的原型芯片验证。代码已开源：https://github.com/SeoLabCornell/torch2chip。