The field of efficient Large Language Model (LLM) inference is rapidly evolving, presenting a unique blend of opportunities and challenges. Although the field has expanded and is vibrant, there hasn't been a concise framework that analyzes the various methods of LLM Inference to provide a clear understanding of this domain. Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques. This framework enables identifying the bottlenecks in LLM deployments and provides a deeper understanding of the practical aspects on real devices, thereby informing more effective strategies for deploying LLM. Furthermore, we systematically collate the latest advancements in efficient LLM inference, covering crucial areas such as weight optimization (e.g., Knowledge Distillation and Quantization), decoding algorithm improvements (e.g., Early Exit and Mixture-of-Expert), and both hardware and system-level enhancements. Distinguished by the integration of roofline model analysis, our survey provides a comprehensive and nuanced exploration of efficient LLM inference challenges and solutions. This distinctive approach not only showcases the current research landscape but also delivers valuable insights for practical implementation, positioning our work as an indispensable resource for researchers new to the field as well as for those seeking to deepen their understanding of efficient LLM deployment. The tool LLM-Viewer is open-sourced.

翻译：高效大语言模型推理领域正在快速发展，呈现出机遇与挑战并存的独特局面。尽管该领域已蓬勃发展且充满活力，但目前尚未有一个简洁框架能够分析各类大语言模型推理方法，从而为这一领域提供清晰的理解。我们的综述不同于传统的文献综述，不仅总结了当前研究现状，还引入了一个基于屋顶线模型的系统分析框架，用于剖析大语言模型推理技术。该框架能够识别大语言模型部署中的瓶颈，并深化对实际设备上应用层面的理解，从而为制定更有效的部署策略提供依据。此外，我们系统梳理了高效大语言模型推理的最新进展，涵盖权重优化（如知识蒸馏与量化）、解码算法改进（如早退与专家混合）以及硬件和系统级增强等关键领域。通过整合屋顶线模型分析，本综述对高效大语言模型推理的挑战与解决方案进行了全面而细致的探讨。这种独特方法不仅展示了当前研究格局，还为实际部署提供了宝贵见解，使我们的工作成为刚踏入该领域的研究者以及希望加深对高效大语言模型部署理解的研究者不可或缺的资源。工具LLM-Viewer已开源。