Prior studies show that adopting the annotation diversity shaped by different backgrounds and life experiences and incorporating them into the model learning, i.e. multi-perspective approach, contribute to the development of more responsible models. Thus, in this paper we propose a new framework for designing and further evaluating perspective-aware models on stance detection task,in which multiple annotators assign stances based on a controversial topic. We also share a new dataset established through obtaining both human and LLM annotations. Results show that the multi-perspective approach yields better classification performance (higher F1-scores), outperforming the traditional approaches that use a single ground-truth, while displaying lower model confidence scores, probably due to the high level of subjectivity of the stance detection task.

翻译：先前的研究表明，采用由不同背景和生活经验形成的标注多样性并将其融入模型学习（即多视角方法），有助于开发更具责任感的模型。因此，本文提出一种新框架，用于设计并进一步评估立场检测任务中的视角感知模型，其中多位标注者基于争议性话题分配立场。我们还分享了一个通过获取人类与LLM标注而构建的新数据集。结果表明，多视角方法能产生更好的分类性能（更高的F1分数），优于使用单一真实标签的传统方法，同时显示出较低的模型置信度，这可能是由于立场检测任务本身具有较高的主观性。