Open Information Extraction (OIE) is a field of natural language processing that aims to present textual information in a format that allows it to be organized, analyzed and reflected upon. Numerous OIE systems are developed, claiming ever-increasing performance, marking the need for objective benchmarks. BenchIE is the latest reference we know of. Despite being very well thought out, we noticed a number of issues we believe are limiting. Therefore, we propose $\textit{BenchIE}^{FL}$, a new OIE benchmark which fully enforces the principles of BenchIE while containing fewer errors, omissions and shortcomings when candidate facts are matched towards reference ones. $\textit{BenchIE}^{FL}$ allows insightful conclusions to be drawn on the actual performance of OIE extractors.
翻译:开放信息抽取(OIE)是自然语言处理的一个领域,其目标是以一种便于组织、分析和反思的格式呈现文本信息。目前已开发出众多OIE系统,其宣称的性能不断提升,这凸显了对客观基准的需求。BenchIE是我们所知的最新参考基准。尽管其设计非常周全,但我们注意到一些我们认为存在限制的问题。因此,我们提出了《BenchIE^{FL}》,这是一个新的OIE基准,它在完全遵循BenchIE原则的同时,在候选事实与参考事实进行匹配时减少了错误、遗漏和缺陷。《BenchIE^{FL}》使得能够对OIE抽取器的实际性能得出有洞察力的结论。