Open knowledge graph (KG) consists of (subject, relation, object) triples extracted from millions of raw text. The subject and object noun phrases and the relation in open KG have severe redundancy and ambiguity and need to be canonicalized. Existing datasets for open KG canonicalization only provide gold entity-level canonicalization for noun phrases. In this paper, we present COMBO, a Complete Benchmark for Open KG canonicalization. Compared with existing datasets, we additionally provide gold canonicalization for relation phrases, gold ontology-level canonicalization for noun phrases, as well as source sentences from which triples are extracted. We also propose metrics for evaluating each type of canonicalization. On the COMBO dataset, we empirically compare previously proposed canonicalization methods as well as a few simple baseline methods based on pretrained language models. We find that properly encoding the phrases in a triple using pretrained language models results in better relation canonicalization and ontology-level canonicalization of the noun phrase. We release our dataset, baselines, and evaluation scripts at https://github.com/jeffchy/COMBO/tree/main.

翻译：开放知识图谱由从海量原始文本中抽取的（主语，关系，宾语）三元组构成。开放知识图谱中的主语和宾语名词短语以及关系存在严重的冗余性和歧义性，需要进行规范化处理。现有的开放知识图谱规范化数据集仅提供名词短语的实体级黄金标准规范化。本文提出COMBO——一个针对开放知识图谱规范化的完整基准。与现有数据集相比，我们额外提供了关系短语的黄金标准规范化、名词短语的本体级黄金标准规范化，以及抽取三元组的源句子。我们还提出了评估每种规范化类型的度量指标。在COMBO数据集上，我们通过实验对比了先前提出的规范化方法以及基于预训练语言模型的若干简单基线方法。研究发现，利用预训练语言模型对三元组中的短语进行恰当编码，能够更有效地实现关系规范化和名词短语的本体级规范化。我们在https://github.com/jeffchy/COMBO/tree/main 发布了本数据集、基线方法及评估脚本。