Knowledge embeddings (KE) represent a knowledge graph (KG) by embedding entities and relations into continuous vector spaces. Existing methods are mainly structure-based or description-based. Structure-based methods learn representations that preserve the inherent structure of KGs. They cannot well represent abundant long-tail entities in real-world KGs with limited structural information. Description-based methods leverage textual information and language models. Prior approaches in this direction barely outperform structure-based ones, and suffer from problems like expensive negative sampling and restrictive description demand. In this paper, we propose LMKE, which adopts Language Models to derive Knowledge Embeddings, aiming at both enriching representations of long-tail entities and solving problems of prior description-based methods. We formulate description-based KE learning with a contrastive learning framework to improve efficiency in training and evaluation. Experimental results show that LMKE achieves state-of-the-art performance on KE benchmarks of link prediction and triple classification, especially for long-tail entities.

翻译：知识嵌入（KE）通过将实体和关系嵌入连续向量空间来表示知识图谱（KG）。现有方法主要基于结构或描述。基于结构的方法学习保留KG固有结构的表示，但难以充分表示现实世界KG中结构信息有限的丰富长尾实体。基于描述的方法利用文本信息和语言模型，但此类先前方法性能仅略优于基于结构的方法，且存在负采样成本高昂、描述需求严格等问题。本文提出LMKE，采用语言模型推导知识嵌入，旨在丰富长尾实体表示并解决先前基于描述方法的缺陷。我们通过对比学习框架构建基于描述的KE学习，提升训练与评估效率。实验结果表明，LMKE在链接预测与三元组分类的KE基准测试中达到最优性能，尤其针对长尾实体表现优异。