In this paper, we propose Latent Relation Language Models (LRLMs), a class of language models that parameterizes the joint distribution over the words in a document and the entities that occur therein via knowledge graph relations. This model has a number of attractive properties: it not only improves language modeling performance, but is also able to annotate the posterior probability of entity spans for a given text through relations. Experiments demonstrate empirical improvements over both a word-based baseline language model and a previous approach that incorporates knowledge graph information. Qualitative analysis further demonstrates the proposed model's ability to learn to predict appropriate relations in context.

翻译：在本文中,我们提出了 " 前关系语言模型 " (LLLMs),这是一组语言模型,对文档中的单词和通过知识图表关系产生的实体进行联合分布的参数,该模型具有若干有吸引力的特性:它不仅改进了语言模型的性能,而且还能够说明实体通过关系跨过某一文本的事后概率。实验表明,在单词基线语言模型和以前纳入知识图表信息的方法上,都取得了经验上的改进。定性分析进一步表明,拟议的模型有能力学习在背景中预测适当关系。