Relation extraction (RE) aims to extract the relations between entity names from the textual context. In principle, textual context determines the ground-truth relation and the RE models should be able to correctly identify the relations reflected by the textual context. However, existing work has found that the RE models memorize the entity name patterns to make RE predictions while ignoring the textual context. This motivates us to raise the question: ``are RE models robust to the entity replacements?'' In this work, we operate the random and type-constrained entity replacements over the RE instances in TACRED and evaluate the state-of-the-art RE models under the entity replacements. We observe the 30\% - 50\% F1 score drops on the state-of-the-art RE models under entity replacements. These results suggest that we need more efforts to develop effective RE models robust to entity replacements. We release the source code at https://github.com/wangywUST/RobustRE.

翻译：关系抽取（RE）旨在从文本上下文中提取实体名称之间的关系。原则上，文本上下文决定真实关系，关系抽取模型应能正确识别文本上下文所反映的关系。然而，现有研究发现关系抽取模型会记忆实体名称模式以进行预测，而忽略文本上下文。这促使我们提出疑问："关系抽取模型对实体替换是否具有鲁棒性？"在本工作中，我们对TACRED中的关系抽取实例执行随机和类型约束的实体替换，并评估当前最先进的关系抽取模型在实体替换下的表现。我们观察到，在实体替换下，最先进的关系抽取模型的F1分数下降了30%-50%。这些结果表明，我们需要投入更多努力来开发对实体替换具有鲁棒性的有效关系抽取模型。我们在https://github.com/wangywUST/RobustRE 发布源代码。