Recent research demonstrates that external knowledge injection can advance pre-trained language models (PLMs) in a variety of downstream NLP tasks. However, existing knowledge injection methods are either applicable to structured knowledge or unstructured knowledge, lacking a unified usage. In this paper, we propose a UNified knowledge inTERface, UNTER, to provide a unified perspective to exploit both structured knowledge and unstructured knowledge. In UNTER, we adopt the decoder as a unified knowledge interface, aligning span representations obtained from the encoder with their corresponding knowledge. This approach enables the encoder to uniformly invoke span-related knowledge from its parameters for downstream applications. Experimental results show that, with both forms of knowledge injected, UNTER gains continuous improvements on a series of knowledge-driven NLP tasks, including entity typing, named entity recognition and relation extraction, especially in low-resource scenarios.

翻译：近期研究表明，外部知识注入可以推进预训练语言模型（PLMs）在多种下游自然语言处理任务中的性能。然而，现有的知识注入方法要么仅适用于结构化知识，要么仅适用于非结构化知识，缺乏统一的用法。本文提出一个统一知识接口UNTER（UNified knowledge inTERface），为同时利用结构化知识和非结构化知识提供统一视角。在UNTER中，我们采用解码器作为统一的知识接口，将从编码器获取的片段（span）表示与其对应知识进行对齐。该方法使编码器能够从其参数中统一调用与片段相关的知识，用于下游应用。实验结果表明，在注入两种形式的知识后，UNTER在一系列知识驱动的NLP任务（包括实体类型识别、命名实体识别和关系抽取）上获得持续改进，尤其在低资源场景下表现显著。