Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows. Such resources include data, time, storage, or energy, all of which are naturally limited and unevenly distributed. This motivates research into efficient methods that require fewer resources to achieve similar results. This survey synthesizes and relates current methods and findings in efficient NLP. We aim to provide both guidance for conducting NLP under limited resources, and point towards promising research directions for developing more efficient methods.

翻译：近期自然语言处理领域的研究通过扩大模型参数和训练数据规模取得了令人瞩目的成果；然而，仅依赖规模提升性能意味着资源消耗也随之增长。这些资源包括数据、时间、存储或能源，它们本质上是有限且分布不均的。这促使研究者探索效率方法——以更少资源实现相近结果。本综述系统整合并关联了当前高效自然语言处理领域的方法与发现。我们旨在为资源受限环境下的NLP研究提供指引，并为开发更高效方法指明有前景的研究方向。