Large Language Models (LLMs) have achieved great success in solving difficult tasks across many domains, but such success comes with a high computation cost, and inference latency. As developers and third parties customize these models, the need to provide efficient inference has increased. Many efforts have attempted to reduce inference cost through model compression techniques such as pruning and distillation. However, these techniques either require labeled data, or are time-consuming as they require the compressed model to be retrained to regain accuracy. In this paper, we propose a gradient-free structured pruning framework that uses only unlabeled data. An evaluation on the GLUE and SQuAD benchmarks using BERT$_{BASE}$ and DistilBERT illustrates the effectiveness of the proposed approach. By only using the weights of the pre-trained model and unlabeled data, in a matter of a few minutes on a single GPU, up to 40% of the original FLOP count can be reduced with less than a 4% accuracy loss across all tasks considered.

翻译：大型语言模型（LLMs）在诸多领域的复杂任务中取得了巨大成功，但此类成功伴随着高昂的计算成本和推理延迟。随着开发者与第三方对这些模型进行定制化调整，高效推理的需求日益增长。现有研究尝试通过剪枝、蒸馏等模型压缩技术降低推理成本，但这些技术或需依赖标注数据，或因需重新训练压缩模型以恢复精度而耗时过长。本文提出一种仅需无标注数据的无梯度结构化剪枝框架。在GLUE和SQuAD基准测试中，使用BERT$_{BASE}$和DistilBERT模型进行的评估验证了该方法的有效性。仅利用预训练模型权重与无标注数据，在单GPU上数分钟内即可将原始FLOP计数降低至多40%，且在所有任务上的精度损失均小于4%。