Knowledge graphs (KGs) have attracted more and more attentions because of their fundamental roles in many tasks. Quality evaluation for KGs is thus crucial and indispensable. Existing methods in this field evaluate KGs by either proposing new quality metrics from different dimensions or measuring performances at KG construction stages. However, there are two major issues with those methods. First, they highly rely on raw data in KGs, which makes KGs' internal information exposed during quality evaluation. Second, they consider more about the quality at data level instead of ability level, where the latter one is more important for downstream applications. To address these issues, we propose a knowledge graph quality evaluation framework under incomplete information (QEII). The quality evaluation task is transformed into an adversarial Q&A game between two KGs. Winner of the game is thus considered to have better qualities. During the evaluation process, no raw data is exposed, which ensures information protection. Experimental results on four pairs of KGs demonstrate that, compared with baselines, the QEII implements a reasonable quality evaluation at ability level under incomplete information.

翻译：知识图谱（KGs）因其在众多任务中的基础性作用而受到越来越多的关注。因此，对知识图谱进行质量评估至关重要且不可或缺。现有方法要么从不同维度提出新的质量指标，要么在知识图谱构建阶段衡量其性能。然而，这些方法存在两个主要问题：第一，它们高度依赖知识图谱中的原始数据，导致质量评估过程中暴露了知识图谱的内部信息；第二，它们更多考虑的是数据层面的质量而非能力层面的质量，而后者对于下游应用更为重要。为解决这些问题，我们提出了一种在不完全信息下进行知识图谱质量评估的框架（QEII）。该框架将质量评估任务转化为两个知识图谱之间的对抗性问答游戏，游戏胜出者即被认为具有更优的质量。在评估过程中，不暴露任何原始数据，从而确保信息保护。在四对知识图谱上的实验结果表明，与基线方法相比，QEII能在不完全信息条件下实现合理的能力层面质量评估。