Large Language Models (LLMs) have demonstrated remarkable capabilities in important tasks such as natural language understanding and language generation, and thus have the potential to make a substantial impact on our society. Such capabilities, however, come with the considerable resources they demand, highlighting the strong need to develop effective techniques for addressing their efficiency challenges. In this survey, we provide a systematic and comprehensive review of efficient LLMs research. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient LLMs topics from model-centric, data-centric, and framework-centric perspective, respectively. We have also created a GitHub repository where we organize the papers featured in this survey at https://github.com/AIoT-MLSys-Lab/Efficient-LLMs-Survey. We will actively maintain the repository and incorporate new research as it emerges. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of efficient LLMs research and inspire them to contribute to this important and exciting field.

翻译：大型语言模型（LLMs）在自然语言理解与生成等重要任务中展现出卓越能力，因而具备对社会产生深远影响的潜力。然而，此类能力的实现伴随着巨大的资源需求，这凸显了开发有效技术以应对其效率挑战的迫切性。本综述对高效大型语言模型研究进行了系统而全面的梳理。我们采用包含三大主类的分类体系组织文献，分别从模型中心、数据中心和框架中心三个既相互独立又彼此关联的视角涵盖高效大型语言模型的核心议题。我们同步创建了GitHub知识库（https://github.com/AIoT-MLSys-Lab/Efficient-LLMs-Survey），系统整理了本综述所涵盖的研究文献。我们将持续维护该知识库并纳入最新研究成果。期望本综述能为研究人员与实践者提供系统理解高效大型语言模型研究的宝贵资源，并激发他们为这一重要且充满活力的领域作出贡献。