Natural Language Processing (NLP) has been revolutionized by the use of Pre-trained Language Models (PLMs) such as BERT. Despite setting new records in nearly every NLP task, PLMs still face a number of challenges including poor interpretability, weak reasoning capability, and the need for a lot of expensive annotated data when applied to downstream tasks. By integrating external knowledge into PLMs, \textit{\underline{K}nowledge-\underline{E}nhanced \underline{P}re-trained \underline{L}anguage \underline{M}odels} (KEPLMs) have the potential to overcome the above-mentioned limitations. In this paper, we examine KEPLMs systematically through a series of studies. Specifically, we outline the common types and different formats of knowledge to be integrated into KEPLMs, detail the existing methods for building and evaluating KEPLMS, present the applications of KEPLMs in downstream tasks, and discuss the future research directions. Researchers will benefit from this survey by gaining a quick and comprehensive overview of the latest developments in this field.

翻译：自然语言处理(NLP)由于使用诸如BERT等预先培训语言模型(PLP)而发生了革命性的变化。尽管在几乎每一项国家语言模型任务中都建立了新记录,但PLM仍然面临许多挑战,包括解释能力差、推理能力薄弱、在下游任务中需要大量昂贵的附加说明数据。具体地说,我们通过将外部知识纳入PLM,\ textitit=underline{K}nowledge-deline{E}加强现有方法,介绍KEPLMS在下游任务中的应用情况,并讨论未来的研究方向。研究者将受益于这项调查,快速全面地了解该领域的最新发展。