The field of efficient Large Language Model (LLM) inference is rapidly evolving, presenting a unique blend of opportunities and challenges. Although the field has expanded and is vibrant, there hasn't been a concise framework that analyzes the various methods of LLM Inference to provide a clear understanding of this domain. Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques. This framework identifies the bottlenecks when deploying LLMs on hardware devices and provides a clear understanding of practical problems, such as why LLMs are memory-bound, how much memory and computation they need, and how to choose the right hardware. We systematically collate the latest advancements in efficient LLM inference, covering crucial areas such as model compression (e.g., Knowledge Distillation and Quantization), algorithm improvements (e.g., Early Exit and Mixture-of-Expert), and both hardware and system-level enhancements. Our survey stands out by analyzing these methods with roofline model, helping us understand their impact on memory access and computation. This distinctive approach not only showcases the current research landscape but also delivers valuable insights for practical implementation, positioning our work as an indispensable resource for researchers new to the field as well as for those seeking to deepen their understanding of efficient LLM deployment. The analyze tool, LLM-Viewer, is open-sourced.

翻译：高效大规模语言模型(LLM)推理领域正快速发展，呈现出机遇与挑战并存的独特格局。尽管该领域已蓬勃发展且充满活力，但目前尚缺乏一个简洁框架来分析LLM推理的各种方法，以提供对该领域的清晰理解。本综述不同于传统文献综述，不仅总结了当前研究状况，还引入了一个基于屋顶线模型的框架，用于系统分析LLM推理技术。该框架能识别在硬件设备上部署LLM时的瓶颈，并提供对实际问题（如为何LLM受内存限制、所需内存和计算量多大、以及如何选择合适硬件）的清晰理解。我们系统汇编了高效LLM推理的最新进展，涵盖模型压缩（如知识蒸馏和量化）、算法改进（如早退法和混合专家模型）以及硬件和系统级增强等关键领域。本综述的特色在于使用屋顶线模型分析这些方法，帮助我们理解它们对内存访问和计算的影响。这一独特方法不仅展示了当前研究图景，还为实际部署提供了宝贵见解，使我们的工作成为新手研究人员以及希望深入理解高效LLM部署的学者不可或缺的资源。分析工具LLM-Viewer已开源。