Structured width pruning of GLU-MLP layers, guided by the Maximum Absolute Weight (MAW) criterion, reveals a systematic dichotomy in how reducing the expansion ratio affects different model capabilities. While performance on tasks relying on parametric knowledge (e.g., MMLU, GSM8K) and perplexity metrics degrades predictably, instruction-following capabilities improve substantially (+46% to +75% in IFEval for Llama-3.2-1B and 3B models), and multi-step reasoning remains robust (MUSR). This pattern challenges the prevailing assumption that pruning induces uniform degradation. We evaluated seven expansion ratio configurations using comprehensive benchmarks assessing factual knowledge, mathematical reasoning, language comprehension, instruction-following, and truthfulness. Our analysis identifies the expansion ratio as a critical architectural parameter that selectively modulates cognitive capabilities, rather than merely serving as a compression metric. We provide the first systematic characterization of this selective preservation phenomenon. Notably, we document a robust inverse correlation (r = -0.864, p = 0.012 in Llama-3B) between factual knowledge capacity (MMLU) and truthfulness metrics (TruthfulQA-MC2): as knowledge degrades, the model's ability to discriminate misconceptions improves consistently. This connects two previously distinct research areas, demonstrating that MAW-guided width pruning acts as a selective filter, reducing parametric knowledge while preserving or enhancing behavioral alignment. Additionally, we quantify context-dependent efficiency trade-offs: pruned configurations achieve up to 23% reduction in energy consumption (J/token) but incur penalties in single-request latency, whereas batch processing workloads benefit uniformly.

翻译：基于最大绝对权重准则引导的GLU-MLP层结构化宽度剪枝揭示了一个系统性二分现象：降低扩展比对不同模型能力的影响方式存在差异。虽然依赖参数化知识的任务性能（如MMLU、GSM8K）和困惑度指标会按预期下降，但指令跟随能力却显著提升（在Llama-3.2-1B和3B模型的IFEval中提升+46%至+75%），多步推理能力保持稳健。这一模式挑战了剪枝会导致性能均匀下降的主流假设。我们使用评估事实知识、数学推理、语言理解、指令跟随和真实性的综合基准测试了七种扩展比配置。我们的分析表明，扩展比是一个关键的架构参数，能够选择性地调节认知能力，而不仅仅是作为压缩指标。我们首次系统性地描述了这种选择性保留现象。值得注意的是，我们记录到事实知识能力与真实性指标之间存在稳健的负相关关系：随着知识退化，模型辨别错误观念的能力持续提升。这连接了两个先前独立的研究领域，表明MAW引导的宽度剪枝充当了选择性过滤器，在减少参数化知识的同时保留或增强了行为对齐。此外，我们量化了上下文相关的效率权衡：剪枝配置可实现高达23%的能耗降低，但在单请求延迟方面会受损失，而批处理工作负载则能普遍受益。