Large Language Models (LLMs) exhibit strong reasoning abilities, but their high computational costs limit their practical deployment. Recent studies reveal significant redundancy in LLMs layers, making layer pruning an active research topic. Layer pruning research primarily focuses on two aspects: measuring layer importance and recovering performance after pruning. Unfortunately, the present works fail to simultaneously maintain pruning performance and efficiency. In this study, we propose GradMAP, a faster layer pruning method with \textbf{Grad}ient \textbf{M}etric \textbf{A}nd \textbf{P}rojection compensation, which consists of two stages. In the first stage, we introduce a novel metric based on gradient magnitudes, enabling a global assessment of layer importance. Note that, it requires only a single backward propagation step per pruning decision, substantially enhancing pruning efficiency. In the second stage, we first analyze the layers with the largest mean shift resulting from pruning, and then incorporate a simple yet effective projection compensation matrix to correct this drift in one step. In this way, the degradation of model performance caused by layer pruning is effectively alleviated. Extensive experiments show that GradMAP outperforms previous layer pruning methods in both pruning speed (achieving an average $4\times$ speedup) and performance.

翻译：大语言模型（LLMs）展现出强大的推理能力，但其高昂的计算成本限制了实际部署。近期研究表明，大语言模型的层中存在显著冗余，使得层剪枝成为一个活跃的研究课题。层剪枝研究主要聚焦于两个方面：衡量层的重要性以及剪枝后的性能恢复。遗憾的是，现有工作未能同时保持剪枝性能与效率。在本研究中，我们提出了GradMAP，一种基于**梯度度量与投影补偿**的快速层剪枝方法，该方法包含两个阶段。在第一阶段，我们引入了一种基于梯度幅度的新型度量指标，能够对层重要性进行全局评估。值得注意的是，该方法每个剪枝决策仅需一次反向传播步骤，从而显著提升了剪枝效率。在第二阶段，我们首先分析因剪枝导致均值偏移最大的层，然后引入一个简单而有效的投影补偿矩阵来一步校正这种偏移。通过这种方式，层剪枝引起的模型性能下降得到了有效缓解。大量实验表明，GradMAP在剪枝速度（平均实现$4\times$加速）和性能方面均优于先前的层剪枝方法。