Relation extraction is essentially a text classification problem, which can be tackled by fine-tuning a pre-trained language model (LM). However, a key challenge arises from the fact that relation extraction cannot straightforwardly be reduced to sequence or token classification. Existing approaches therefore solve the problem in an indirect way: they fine-tune an LM to learn embeddings of the head and tail entities, and then predict the relationship from these entity embeddings. Our hypothesis in this paper is that relation extraction models can be improved by capturing relationships in a more direct way. In particular, we experiment with appending a prompt with a [MASK] token, whose contextualised representation is treated as a relation embedding. While, on its own, this strategy significantly underperforms the aforementioned approach, we find that the resulting relation embeddings are highly complementary to what is captured by embeddings of the head and tail entity. By jointly considering both types of representations, we end up with a simple model that outperforms the state-of-the-art across several relation extraction benchmarks.

翻译：关系抽取本质上是一个文本分类问题，可以通过微调预训练语言模型（LM）来解决。然而，一个关键挑战在于关系抽取无法直接简化为序列或词元分类。现有方法因此以间接方式解决该问题：通过微调LM学习头实体和尾实体的嵌入表示，然后基于这些实体嵌入预测关系。本文假设通过更直接的方式捕捉关系信息可以改进关系抽取模型。具体而言，我们尝试在输入后附加包含[MASK]标记的提示模板，并将其上下文表示作为关系嵌入。虽然单独使用该策略的性能显著低于前述方法，但我们发现由此得到的关系嵌入与头尾实体嵌入所捕获的信息具有高度互补性。通过联合考虑这两种表示类型，我们最终构建了一个简单模型，在多个关系抽取基准测试中超越了现有最优方法。