Diverse Natural Language Processing tasks employ constituency parsing to understand the syntactic structure of a sentence according to a phrase structure grammar. Many state-of-the-art constituency parsers are proposed, but they may provide different results for the same sentences, especially for corpora outside their training domains. This paper adopts the truth discovery idea to aggregate constituency parse trees from different parsers by estimating their reliability in the absence of ground truth. Our goal is to consistently obtain high-quality aggregated constituency parse trees. We formulate the constituency parse tree aggregation problem in two steps, structure aggregation and constituent label aggregation. Specifically, we propose the first truth discovery solution for tree structures by minimizing the weighted sum of Robinson-Foulds (RF) distances, a classic symmetric distance metric between two trees. Extensive experiments are conducted on benchmark datasets in different languages and domains. The experimental results show that our method, CPTAM, outperforms the state-of-the-art aggregation baselines. We also demonstrate that the weights estimated by CPTAM can adequately evaluate constituency parsers in the absence of ground truth.

翻译：各类自然语言处理任务常利用成分句法分析（constituency parse）基于短语结构语法理解句子的句法结构。现有许多先进的成分句法分析器，但它们在处理相同句子时可能产生不同结果，尤其是对训练领域之外的语料。本文采用真值发现（truth discovery）思想，在缺乏真值标注的情况下通过估计不同分析器的可靠性来聚合成分句法树，目标是一致地获得高质量的聚合成分句法树。我们将成分句法树聚合问题分解为结构聚合与成分标签聚合两个步骤。具体而言，我们首次提出针对树结构的真值发现解决方案，通过最小化Robinson-Foulds（RF）距离（一种经典的树间对称距离度量）的加权和实现。在跨语言及跨领域的基准数据集上进行了广泛实验，结果表明我们的方法CPTAM优于现有最先进的聚合基线方法。我们还证明，在无真值情况下，CPTAM估计的权重能够有效评估成分句法分析器。