The field of efficient Large Language Model (LLM) inference is rapidly evolving, presenting a unique blend of opportunities and challenges. Although the field has expanded and is vibrant, there hasn't been a concise framework that analyzes the various methods of LLM Inference to provide a clear understanding of this domain. Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques. This framework identifies the bottlenecks when deploying LLMs on hardware devices and provides a clear understanding of practical problems, such as why LLMs are memory-bound, how much memory and computation they need, and how to choose the right hardware. We systematically collate the latest advancements in efficient LLM inference, covering crucial areas such as model compression (e.g., Knowledge Distillation and Quantization), algorithm improvements (e.g., Early Exit and Mixture-of-Expert), and both hardware and system-level enhancements. Our survey stands out by analyzing these methods with roofline model, helping us understand their impact on memory access and computation. This distinctive approach not only showcases the current research landscape but also delivers valuable insights for practical implementation, positioning our work as an indispensable resource for researchers new to the field as well as for those seeking to deepen their understanding of efficient LLM deployment. The analyze tool, LLM-Viewer, is open-sourced.

翻译：高效大语言模型（LLM）推理领域正在快速发展，呈现出机遇与挑战并存的独特格局。尽管该领域已蓬勃发展，但目前仍缺乏一个能够系统分析各种LLM推理方法并清晰揭示其内涵的简明框架。本综述不同于传统文献回顾，不仅总结了当前研究进展，还引入基于屋顶线模型的分析框架，用于系统剖析LLM推理技术。该框架能够识别在硬件设备上部署LLM时的性能瓶颈，并清晰解答实际问题，例如为何LLM受内存限制、它们需要多少内存与计算资源，以及如何选择合适的硬件。我们系统梳理了高效LLM推理的最新进展，涵盖模型压缩（如知识蒸馏与量化）、算法改进（如早退机制与混合专家模型）以及硬件与系统级优化等关键领域。本综述的独到之处在于运用屋顶线模型分析这些方法，深入理解其对内存访问与计算的影响。这种独特方法不仅展示了当前研究全景，还为实际部署提供了宝贵见解，使本文成为初入该领域的研究者以及希望深入理解高效LLM部署的学者不可或缺的参考资源。分析工具LLM-Viewer已开源。