Large, pre-trained transformer-based language models such as BERT have drastically changed the Natural Language Processing (NLP) field. We present a survey of recent work that uses these large language models to solve NLP tasks via pre-training then fine-tuning, prompting, or text generation approaches. We also present approaches that use pre-trained language models to generate data for training augmentation or other purposes. We conclude with discussions on limitations and suggested directions for future research.

翻译：诸如BERT等大型、以培训前变压器为基础的变压器语言模型极大地改变了自然语言处理(NLP)领域。我们介绍了对最近工作的调查,即利用这些大语言模型通过培训前和微调、推动或文本生成方法解决非语言处理任务。我们还介绍了使用培训前语言模型生成数据用于培训扩增或其他目的的方法。我们最后讨论了局限性和今后研究的拟议方向。