Large Language Models (LLMs) have demonstrated remarkable capabilities in important tasks such as natural language understanding, language generation, and complex reasoning and have the potential to make a substantial impact on our society. Such capabilities, however, come with the considerable resources they demand, highlighting the strong need to develop effective techniques for addressing their efficiency challenges.In this survey, we provide a systematic and comprehensive review of efficient LLMs research. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient LLMs topics from model-centric, data-centric, and framework-centric perspective, respectively. We have also created a GitHub repository where we compile the papers featured in this survey at https://github.com/AIoT-MLSys-Lab/Efficient-LLMs-Survey, and will actively maintain this repository and incorporate new research as it emerges. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of the research developments in efficient LLMs and inspire them to contribute to this important and exciting field.

翻译：大型语言模型（LLMs）在自然语言理解、语言生成和复杂推理等重要任务中展现出卓越能力，并有望对社会产生深远影响。然而，这些能力伴随着巨大的资源需求，凸显了开发有效技术以应对其效率挑战的迫切性。本综述对高效LLMs研究进行了系统而全面的回顾。我们提出了一种包含三大类别的分类体系，分别从模型中心、数据中心和框架中心视角，系统梳理了高效LLMs领域内相互关联的不同研究方向。我们还创建了GitHub仓库（https://github.com/AIoT-MLSys-Lab/Efficient-LLMs-Survey）用于汇总本综述收录的论文，并将持续维护该仓库，及时纳入最新研究成果。希望本综述能够帮助研究人员和从业者系统理解高效LLMs领域的研究进展，并激励他们为这一重要且激动人心的领域做出贡献。