Sparse attention accelerates Diffusion Transformers (DiTs) for video generation by computing only the important tokens while skipping the rest. The token selection strategy is key to balancing sparsity and accuracy. We formulate the token filtering process as a dual-goal optimization problem: maximizing sparsity and minimizing accuracy degradation. Existing algorithms cannot fulfill both objectives simultaneously. For example, Top-p only considers the accuracy constraint, while Top-k maintains a fixed computational budget but loosens the accuracy constraint. This paper demonstrates that maintaining a fixed recall rate is sufficient for ensuring accuracy, whereas a fixed threshold is suboptimal for reducing computational cost. Therefore, we propose a dynamic thresholding scheme to improve sparsity while maintaining the same level of accuracy. Furthermore, our algorithm is deeply integrated with Flash Attention (FA), eliminating the need for any additional masking computation overhead. Experimental results on Wan 2.2 validate that, compared to the BLASST algorithm which is also integrated with FA, our dynamic thresholding strategy enhances sparsity from 61.42\% to 82\% with a VBench metric drop of less than 5\%. This results in an approximate 15\% in attention computation and a $1.61\times$ increase in computational efficiency, which is 1.18x higher than that of BLASST.

翻译：稀疏注意力通过仅计算关键令牌而跳过其余部分，加速了用于视频生成的扩散变换器（DiTs）。令牌选择策略是平衡稀疏性和准确性的关键。本文将令牌过滤过程表述为一个双目标优化问题：最大化稀疏性和最小化准确性下降。现有算法无法同时实现这两个目标。例如，Top-p仅考虑准确性约束，而Top-k保持固定的计算预算但放松了准确性约束。本文证明，保持固定的召回率足以确保准确性，而固定的阈值对于降低计算成本而言是次优的。因此，我们提出了一种动态阈值方案，以在保持相同准确性水平的同时提高稀疏性。此外，我们的算法与Flash Attention（FA）深度集成，无需任何额外的掩码计算开销。在Wan 2.2上的实验结果表明，与同样集成FA的BLASST算法相比，我们的动态阈值策略将稀疏性从61.42%提升到82%，同时VBench指标下降小于5%。这使注意力计算减少约15%，计算效率提升1.61倍，比BLASST高出1.18倍。