Large Language Models (LLMs) have demonstrated remarkable capabilities in important tasks such as natural language understanding, language generation, and complex reasoning and have the potential to make a substantial impact on our society. Such capabilities, however, come with the considerable resources they demand, highlighting the strong need to develop effective techniques for addressing their efficiency challenges. In this survey, we provide a systematic and comprehensive review of efficient LLMs research. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient LLMs topics from model-centric, data-centric, and framework-centric perspective, respectively. We have also created a GitHub repository where we compile the papers featured in this survey at https://github.com/AIoT-MLSys-Lab/EfficientLLMs, https://github.com/AIoT-MLSys-Lab/Efficient-LLMs-Survey, and will actively maintain this repository and incorporate new research as it emerges. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of the research developments in efficient LLMs and inspire them to contribute to this important and exciting field.

翻译：大规模语言模型（LLMs）在自然语言理解、语言生成和复杂推理等重要任务中展现出卓越能力，并有望对社会产生深远影响。然而，这些能力伴随着巨大的资源需求，凸显了开发有效技术以解决其效率挑战的迫切性。本综述对高效LLMs研究进行了系统全面的梳理。我们提出一个包含三个主类别的分类体系，分别从模型中心、数据中心和框架中心视角，覆盖了相互关联的高效LLMs主题。我们还创建了GitHub仓库（https://github.com/AIoT-MLSys-Lab/EfficientLLMs，https://github.com/AIoT-MLSys-Lab/Efficient-LLMs-Survey），收录本综述引用的论文，并将持续维护该仓库，纳入新涌现的研究成果。期待本综述能成为宝贵资源，帮助研究者和从业者系统理解高效LLMs的研究进展，并激励他们为这一重要且激动人心的领域做出贡献。