Subjectivity and difference of opinion are key social phenomena, and it is crucial to take these into account in the annotation and detection process of derogatory textual content. In this paper, we use four datasets provided by SemEval-2023 Task 11 and fine-tune a BERT model to capture the disagreement in the annotation. We find individual annotator modeling and aggregation lowers the Cross-Entropy score by an average of 0.21, compared to the direct training on the soft labels. Our findings further demonstrate that annotator metadata contributes to the average 0.029 reduction in the Cross-Entropy score.

翻译：主观性和意见分歧是关键的社会现象，在贬损性文本内容的标注与检测过程中必须充分考虑这些因素。本文使用SemEval-2023任务11提供的四个数据集，通过微调BERT模型来捕捉标注过程中的分歧。我们发现，与基于软标签的直接训练相比，个体标注者建模与聚合方法使交叉熵得分平均降低0.21。进一步研究表明，标注者元数据可使交叉熵得分平均降低0.029。