This paper presents results of our system for CoMeDi Shared Task, focusing on Subtask 2: Disagreement Ranking. Our system leverages sentence embeddings generated by the paraphrase-xlm-r-multilingual-v1 model, combined with a deep neural regression model incorporating batch normalization and dropout for improved generalization. By predicting the mean of pairwise judgment differences between annotators, our method explicitly targets disagreement ranking, diverging from traditional "gold label" aggregation approaches. We optimized our system with a customized architecture and training procedure, achieving competitive performance in Spearman correlation against mean disagreement labels. Our results highlight the importance of robust embeddings, effective model architecture, and careful handling of judgment differences for ranking disagreement in multilingual contexts. These findings provide insights into the use of contextualized representations for ordinal judgment tasks and open avenues for further refinement of disagreement prediction models.
翻译:本文介绍了我们为 CoMeDi 共享任务所开发系统的结果,重点关注子任务 2:分歧排序。我们的系统利用 paraphrase-xlm-r-multilingual-v1 模型生成的句子嵌入,并结合了包含批归一化与丢弃法的深度神经回归模型以提升泛化能力。通过预测标注者之间成对判断差异的均值,我们的方法明确针对分歧排序,有别于传统的“黄金标签”聚合方法。我们通过定制化的架构与训练流程优化了系统,在针对平均分歧标签的斯皮尔曼相关性上取得了有竞争力的性能。我们的结果突显了在多语言语境下,用于分歧排序的鲁棒嵌入、有效的模型架构以及对判断差异的谨慎处理的重要性。这些发现为在序数判断任务中使用上下文表征提供了见解,并为分歧预测模型的进一步改进开辟了途径。