We focus on addressing the dense backward propagation issue for training efficiency of N:M fine-grained sparsity that preserves at most N out of M consecutive weights and achieves practical speedups supported by the N:M sparse tensor core. Therefore, we present a novel method of Bi-directional Masks (Bi-Mask) with its two central innovations in: 1) Separate sparse masks in the two directions of forward and backward propagation to obtain training acceleration. It disentangles the forward and backward weight sparsity and overcomes the very dense gradient computation. 2) An efficient weight row permutation method to maintain performance. It picks up the permutation candidate with the most eligible N:M weight blocks in the backward to minimize the gradient gap between traditional uni-directional masks and our bi-directional masks. Compared with existing uni-directional scenario that applies a transposable mask and enables backward acceleration, our Bi-Mask is experimentally demonstrated to be more superior in performance. Also, our Bi-Mask performs on par with or even better than methods that fail to achieve backward acceleration. Project of this paper is available at \url{https://github.com/zyxxmu/Bi-Mask}.

翻译：本文聚焦于解决N:M细粒度稀疏性在训练效率中面临的密集反向传播问题——该稀疏性在M个连续权重中最多保留N个非零权重，并借助N:M稀疏张量核心实现实际加速。为此，我们提出一种名为双向掩码（Bi-Mask）的新颖方法，其核心创新包括：1) 在前向与反向传播两个方向上分别应用独立稀疏掩码以实现训练加速。该方法解耦了前向与反向的权重稀疏性，克服了密集梯度计算问题。2) 一种高效的权重行重排方法以维持性能。该方法在反向传播中选取包含最合格N:M权重块的排列候选方案，从而最小化传统单向掩码与双向掩码之间的梯度差异。与现有采用可转置掩码并实现反向加速的单向方案相比，实验表明我们的Bi-Mask在性能上更具优势。此外，Bi-Mask的表现与尚未实现反向加速的方法相当甚至更优。本文项目开源代码见 \url{https://github.com/zyxxmu/Bi-Mask}。