Transformer based large language models have achieved tremendous success. However, the significant memory and computational costs incurred during the inference process make it challenging to deploy large models on resource-constrained devices. In this paper, we investigate compression and efficient inference methods for large language models from an algorithmic perspective. Regarding taxonomy, similar to smaller models, compression and acceleration algorithms for large language models can still be categorized into quantization, pruning, distillation, compact architecture design, dynamic networks. However, Large language models have two prominent characteristics compared to smaller models: (1) Most of compression algorithms require finetuning or even retraining the model after compression. The most notable aspect of large models is the very high cost associated with model finetuning or training. Therefore, many algorithms for large models, such as quantization and pruning, start to explore tuning-free algorithms. (2) Large models emphasize versatility and generalization rather than performance on a single task. Hence, many algorithms, such as knowledge distillation, focus on how to preserving their versatility and generalization after compression. Since these two characteristics were not very pronounced in early large models, we further distinguish large language models into medium models and ``real'' large models. Additionally, we also provide an introduction to some mature frameworks for efficient inference of large models, which can support basic compression or acceleration algorithms, greatly facilitating model deployment for users.

翻译：基于Transformer的大型语言模型取得了巨大成功。然而，推理过程中产生的大量内存和计算开销使得在资源受限设备上部署大型模型面临挑战。本文从算法角度研究大型语言模型的压缩与高效推理方法。在分类方面，与较小模型类似，大型语言模型的压缩与加速算法仍可归类为量化、剪枝、蒸馏、紧凑架构设计、动态网络。但大型语言模型与较小模型相比有两个显著特征：(1) 大多数压缩算法在压缩后需要微调甚至重新训练模型。大型模型最显著的特点是模型微调或训练成本极高。因此，许多针对大模型的算法（如量化和剪枝）开始探索免调优算法。(2) 大型模型注重通用性和泛化能力而非单一任务性能。因此，许多算法（如知识蒸馏）关注如何在压缩后保持其通用性和泛化能力。由于这两个特征在早期大模型中并不显著，我们进一步将大型语言模型区分为中型模型和"真正"大型模型。此外，我们还介绍了一些成熟的模型高效推理框架，这些框架可支持基本的压缩或加速算法，极大地方便用户的模型部署。