The emergence of Transformer-based Large Language Models (LLMs) has substantially augmented the capabilities of Natural Language Processing (NLP), thereby intensifying the demand for computational resources. Therefore, enhancing efficiency based on factors like computational requirements, energy consumption, carbon footprint and financial cost has become a vital area of research. This motivates us to conduct a systematic literature review on Transformer-based LLMs in NLP from the perspective of efficiency. In this survey of 312 articles published between the years 2011 and 2025, efficiency-improvement endeavors have been systematically discussed targeting various aspects such as data curation, model design, model downsizing, and dynamic inferencing. This has been augmented with efficiency considerations in model adaptation strategies like pre-training, fine-tuning, prompt-engineering and Retrieval-Augmented Generation (RAG). Furthermore, a statistical analysis of the articles has been performed followed by an in-depth evaluation of the efficiency and efficacy of more than 30 renowned NLP models has been conducted on 13 evaluation benchmarks. This paper offers valuable insights for researchers, professionals as well as scholars, and explores the trend of research toward sustainable practices in NLP.

翻译：基于Transformer的大语言模型（LLM）的出现显著增强了自然语言处理（NLP）的能力，同时也加剧了对计算资源的需求。因此，基于计算需求、能耗、碳足迹及财务成本等因素提升效率已成为一个至关重要的研究领域。这促使我们从效率的视角出发，对NLP中基于Transformer的LLM进行系统的文献综述。本综述涵盖了2011年至2025年间发表的312篇文献，系统性地讨论了针对数据策展、模型设计、模型小型化及动态推理等多个方面的效率提升工作，并进一步结合了预训练、微调、提示工程及检索增强生成（RAG）等模型适应策略中的效率考量。此外，我们对相关文献进行了统计分析，并在13个评估基准上对超过30个知名NLP模型的效率与效能进行了深入评估。本文为研究人员、从业者及学者提供了有价值的见解，并探讨了NLP领域向可持续实践发展的研究趋势。