The field of efficient Large Language Model (LLM) inference is rapidly evolving, presenting a unique blend of opportunities and challenges. Although the field has expanded and is vibrant, there hasn't been a concise framework that analyzes the various methods of LLM Inference to provide a clear understanding of this domain. Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques. This framework identifies the bottlenecks when deploying LLMs on hardware devices and provides a clear understanding of practical problems, such as why LLMs are memory-bound, how much memory and computation they need, and how to choose the right hardware. We systematically collate the latest advancements in efficient LLM inference, covering crucial areas such as model compression (e.g., Knowledge Distillation and Quantization), algorithm improvements (e.g., Early Exit and Mixture-of-Expert), and both hardware and system-level enhancements. Our survey stands out by analyzing these methods with roofline model, helping us understand their impact on memory access and computation. This distinctive approach not only showcases the current research landscape but also delivers valuable insights for practical implementation, positioning our work as an indispensable resource for researchers new to the field as well as for those seeking to deepen their understanding of efficient LLM deployment. The analyze tool LLM-Viewer is open-sourced.

翻译：高效大型语言模型（LLM）推理领域正在快速发展，呈现出机遇与挑战并存的独特格局。尽管该领域已蓬勃发展且充满活力，但目前仍缺乏一个简洁框架来分析LLM推理的多种方法，以提供对该领域的清晰理解。本综述区别于传统文献综述，不仅总结当前研究现状，还引入基于屋顶线模型（roofline model）的框架，用于系统分析LLM推理技术。该框架能够识别在硬件设备上部署LLM时的瓶颈，并提供对实际问题（例如LLM为何受内存限制、需要多少内存与计算资源、以及如何选择合适的硬件）的清晰理解。我们系统梳理了高效LLM推理的最新进展，涵盖模型压缩（如知识蒸馏与量化）、算法改进（如早退机制与混合专家模型）以及硬件与系统级优化等关键领域。本综述的独特之处在于利用屋顶线模型分析这些方法，从而揭示其对内存访问与计算的影响。这一独特方法不仅展示了当前研究格局，还为实际部署提供了宝贵见解，使本工作成为初入该领域的研究者以及希望深入理解高效LLM部署的学者不可或缺的资源。分析工具LLM-Viewer已开源。