The field of efficient Large Language Model (LLM) inference is rapidly evolving, presenting a unique blend of opportunities and challenges. Although the field has expanded and is vibrant, there hasn't been a concise framework that analyzes the various methods of LLM Inference to provide a clear understanding of this domain. Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques. This framework identifies the bottlenecks when deploying LLMs on hardware devices and provides a clear understanding of practical problems, such as why LLMs are memory-bound, how much memory and computation they need, and how to choose the right hardware. We systematically collate the latest advancements in efficient LLM inference, covering crucial areas such as model compression (e.g., Knowledge Distillation and Quantization), algorithm improvements (e.g., Early Exit and Mixture-of-Expert), and both hardware and system-level enhancements. Our survey stands out by analyzing these methods with roofline model, helping us understand their impact on memory access and computation. This distinctive approach not only showcases the current research landscape but also delivers valuable insights for practical implementation, positioning our work as an indispensable resource for researchers new to the field as well as for those seeking to deepen their understanding of efficient LLM deployment. The analyze tool, LLM-Viewer, is open-sourced.

翻译：高效大型语言模型（LLM）推理领域正在快速发展，呈现出机遇与挑战并存的独特格局。尽管该领域已蓬勃扩张，但仍缺乏一个简洁的框架来分析各种LLM推理方法，以提供对该领域的清晰理解。本综述不同于传统文献回顾，不仅总结了当前研究现状，还引入了一种基于屋顶线模型的系统分析LLM推理技术的框架。该框架能识别在硬件设备上部署LLM时的瓶颈，并清晰揭示实际问题，例如为何LLM受限于内存、所需的内存与计算量以及如何选择合适的硬件。我们系统梳理了高效LLM推理的最新进展，涵盖模型压缩（如知识蒸馏和量化）、算法改进（如早期退出和混合专家）以及硬件与系统层面的优化等关键领域。本综述的独特之处在于利用屋顶线模型分析这些方法，从而揭示它们对内存访问和计算的影响。这种独具特色的方法不仅展现了当前研究全景，还为实际部署提供了宝贵见解，使我们的工作成为领域新手和希望深入理解高效LLM部署的研究人员不可或缺的资源。分析工具LLM-Viewer已开源。