This paper presents a Spatial Re-parameterization (SpRe) method for the N:M sparsity in CNNs. SpRe is stemmed from an observation regarding the restricted variety in spatial sparsity present in N:M sparsity compared with unstructured sparsity. Particularly, N:M sparsity exhibits a fixed sparsity rate within the spatial domains due to its distinctive pattern that mandates N non-zero components among M successive weights in the input channel dimension of convolution filters. On the contrary, we observe that unstructured sparsity displays a substantial divergence in sparsity across the spatial domains, which we experimentally verified to be very crucial for its robust performance retention compared with N:M sparsity. Therefore, SpRe employs the spatial-sparsity distribution of unstructured sparsity to assign an extra branch in conjunction with the original N:M branch at training time, which allows the N:M sparse network to sustain a similar distribution of spatial sparsity with unstructured sparsity. During inference, the extra branch can be further re-parameterized into the main N:M branch, without exerting any distortion on the sparse pattern or additional computation costs. SpRe has achieved a commendable feat by matching the performance of N:M sparsity methods with state-of-the-art unstructured sparsity methods across various benchmarks. Code and models are anonymously available at \url{https://github.com/zyxxmu/SpRe}.

翻译：本文提出了一种名为空间再参数化（SpRe）的方法，用于解决CNN中的N:M稀疏性问题。SpRe源于对N:M稀疏性与非结构化稀疏性在空间稀疏性多样性方面的观察差异。具体而言，由于N:M稀疏性要求卷积滤波器输入通道维度中连续M个权重内包含N个非零分量，其空间域中的稀疏率固定不变。相反，我们观察到非结构化稀疏性在空间域中表现出显著的稀疏性差异，实验证明这一特性对保持其优于N:M稀疏性的稳健性能至关重要。因此，SpRe利用非结构化稀疏性的空间稀疏性分布，在训练阶段为原始N:M分支分配一个额外分支，使N:M稀疏网络能够维持与非结构化稀疏性相似的空间稀疏性分布。推理阶段，该额外分支可进一步再参数化至主N:M分支中，既不影响稀疏模式也不增加计算开销。SpRe在多项基准测试中展现出卓越性能，其表现可与最先进的非结构化稀疏方法相媲美。代码与模型匿名发布于\url{https://github.com/zyxxmu/SpRe}。