Structured pruning is a promising approach to create smaller, faster large language models. However, existing methods typically rely on computing the gradient via backward passes, which can inflate memory requirements and compute costs. In this work we introduce Bonsai, a gradient-free structured pruning method that eliminates the need for backpropagation, significantly reducing memory requirements and compute costs while achieving state-of-the-art pruning performance. Bonsai uses forward-pass-only perturbative pruning to enable efficient compression of large models on a broader range of hardware configurations. Unlike existing structured pruning approaches, Bonsai not only achieves better compression with fewer resources but also produces models that are twice as fast as those generated by semi-structured pruning. As a concrete demonstration, we use Bonsai to prune 7B and 8B models to 50% sparsity on a single A6000 GPU -- a task challenging for backprop-based methods in memory-constrained settings, as they require 2-3x the memory. Our results show that removing backprop as a requirement not only enables pruning larger models on constrained hardware but can also lead to state-of-the-art efficiency and performance.

翻译：结构化剪枝是创建更小、更快大语言模型的一种前景广阔的方法。然而，现有方法通常依赖于通过反向传播计算梯度，这可能会增加内存需求和计算成本。本文提出Bonsai，一种无需梯度的结构化剪枝方法，它消除了对反向传播的需求，在实现先进剪枝性能的同时显著降低了内存需求和计算成本。Bonsai采用仅需前向传播的扰动剪枝技术，使得大模型能够在更广泛的硬件配置上实现高效压缩。与现有结构化剪枝方法不同，Bonsai不仅以更少资源实现更好的压缩效果，且生成的模型速度可达半结构化剪枝所生成模型的两倍。作为具体例证，我们使用Bonsai在单张A6000 GPU上将7B和8B模型剪枝至50%稀疏度——这对于基于反向传播的方法在内存受限环境中具有挑战性，因其通常需要2-3倍内存。我们的结果表明，消除对反向传播的依赖不仅能在受限硬件上实现更大模型的剪枝，还能带来先进的效率与性能表现。