Contrastive pre-training on distant supervision has shown remarkable effectiveness in improving supervised relation extraction tasks. However, the existing methods ignore the intrinsic noise of distant supervision during the pre-training stage. In this paper, we propose a weighted contrastive learning method by leveraging the supervised data to estimate the reliability of pre-training instances and explicitly reduce the effect of noise. Experimental results on three supervised datasets demonstrate the advantages of our proposed weighted contrastive learning approach compared to two state-of-the-art non-weighted baselines.Our code and models are available at: https://github.com/YukinoWan/WCL

翻译：远程监督上的对比预训练在提升有监督关系抽取任务方面展现出显著效果。然而，现有方法在预训练阶段忽略了远程监督固有的噪声问题。本文提出一种加权对比学习方法，利用有监督数据估计预训练实例的可靠性，并显式降低噪声影响。在三个有监督数据集上的实验结果表明，与两种当前最优的非加权基线方法相比，我们提出的加权对比学习方法具有明显优势。我们的代码和模型可在以下链接获取：https://github.com/YukinoWan/WCL