Post-training pruning substantially reduces inference costs but often causes severe quality degradation without adapting the remaining weights. For LLMs, such retraining is commonly considered impractical due to large computational costs, motivating increasingly sophisticated pruning criteria to compensate by selecting better sparsity patterns. In this work, we revisit post-pruning adaptation and study local reconstruction: adapting only a small pruned submodel at a time using a small calibration set by matching intermediate activations of the dense model. We conduct a large-scale study across model families and scales (up to 72B parameters) and establish three central results. First, local reconstruction is an effective adaptation mechanism for LLMs, matching post-pruning PEFT while using over an order of magnitude less data and compute. Second, we identify a broad "free lunch" regime in reconstruction granularity: across a wide range of submodel sizes, final quality remains essentially unchanged, allowing granularity to be chosen based on memory constraints. Finally, with reconstruction, the pruning criterion becomes less critical: performance gaps between sophisticated methods and simple baselines shrink with model size, making simple methods competitive again. Collectively, our results challenge the prevailing narrative that post-pruning adaptation is impractical for LLMs.

翻译：训练后剪枝能显著降低推理成本，但若不调整剩余权重，常导致严重的质量下降。对于大语言模型，此类重训练通常被认为因计算成本巨大而不切实际，这促使人们设计日益复杂的剪枝标准，以期通过选择更优的稀疏模式来补偿。在本工作中，我们重新审视剪枝后的适应过程，并研究局部重构：仅使用小型校准集，通过匹配稠密模型的中间激活，每次仅适应一个小的、已剪枝的子模型。我们在不同模型系列和规模（最高达720亿参数）上进行了大规模研究，并确立了三个核心结果。首先，局部重构是大语言模型一种有效的适应机制，其效果与剪枝后的参数高效微调相当，但使用的数据和计算量少一个数量级以上。其次，我们识别出重构粒度中存在一个广泛的“免费午餐”区间：在很宽的子模型大小范围内，最终质量基本保持不变，从而允许根据内存约束选择粒度。最后，通过重构，剪枝标准的重要性降低：复杂方法与简单基线之间的性能差距随模型规模增大而缩小，使得简单方法重新具备竞争力。总体而言，我们的研究结果挑战了当前普遍认为的、对大语言模型进行剪枝后适应不切实际的看法。