Deep Learning (DL) acceleration support in CPUs has recently gained a lot of traction, with several companies (Arm, Intel, IBM) announcing products with specialized matrix engines accessible via GEMM instructions. CPUs are pervasive and need to handle diverse requirements across DL workloads running in edge/HPC/cloud platforms. Therefore, as DL workloads embrace sparsity to reduce the computations and memory size of models, it is also imperative for CPUs to add support for sparsity to avoid under-utilization of the dense matrix engine and inefficient usage of the caches and registers. This work presents VEGETA, a set of ISA and microarchitecture extensions over dense matrix engines to support flexible structured sparsity for CPUs, enabling programmable support for diverse DL models with varying degrees of sparsity. Compared to the state-of-the-art (SOTA) dense matrix engine in CPUs, a VEGETA engine provides 1.09x, 2.20x, 3.74x, and 3.28x speed-ups when running 4:4 (dense), 2:4, 1:4, and unstructured (95%) sparse DNN layers.

翻译：深度学习（DL）加速支持在CPU中近来受到广泛关注，多家公司（Arm、Intel、IBM）已宣布推出支持GEMM指令的专用矩阵引擎产品。CPU具有普适性，需满足边缘计算/HPC/云平台上不同DL工作负载的多样化需求。随着DL工作负载引入稀疏性以降低模型计算量与存储空间，CPU必须增加对稀疏性的支持，以避免稠密矩阵引擎利用率不足以及缓存与寄存器的低效使用。本文提出VEGETA——一系列在稠密矩阵引擎基础上的ISA与微架构扩展，支持CPU的灵活结构化稀疏性，可编程适配不同稀疏度的多样化DL模型。与CPU中最先进的稠密矩阵引擎相比，VEGETA引擎在运行4:4（稠密）、2:4、1:4及非结构化（95%稀疏）深度神经网络层时，分别实现1.09倍、2.20倍、3.74倍和3.28倍的加速比。