Argument retrieval is the task of finding relevant arguments for a given query. While existing approaches rely solely on the semantic alignment of queries and arguments, this first shared task on perspective argument retrieval incorporates perspectives during retrieval, accounting for latent influences in argumentation. We present a novel multilingual dataset covering demographic and socio-cultural (socio) variables, such as age, gender, and political attitude, representing minority and majority groups in society. We distinguish between three scenarios to explore how retrieval systems consider explicitly (in both query and corpus) and implicitly (only in query) formulated perspectives. This paper provides an overview of this shared task and summarizes the results of the six submitted systems. We find substantial challenges in incorporating perspectivism, especially when aiming for personalization based solely on the text of arguments without explicitly providing socio profiles. Moreover, retrieval systems tend to be biased towards the majority group but partially mitigate bias for the female gender. While we bootstrap perspective argument retrieval, further research is essential to optimize retrieval systems to facilitate personalization and reduce polarization.
翻译:论据检索的任务是为给定查询寻找相关论据。现有方法仅依赖于查询与论据的语义对齐,而首个视角化论据检索共享任务在检索过程中引入了视角,以考量论证中潜在的隐性影响。我们提出了一个新颖的多语言数据集,涵盖人口统计学和社会文化(社会)变量,如年龄、性别和政治态度,代表了社会中的少数与多数群体。我们区分了三种场景,以探索检索系统如何考量显式(在查询和语料库中均明确)和隐式(仅在查询中隐含)表述的视角。本文概述了该共享任务,并总结了提交的六个系统的结果。我们发现,在融入视角化方面存在显著挑战,尤其是在仅基于论据文本而不明确提供社会属性档案的情况下追求个性化检索时。此外,检索系统往往偏向于多数群体,但在女性性别方面部分缓解了这种偏见。虽然我们启动了视角化论据检索的研究,但进一步的研究对于优化检索系统以促进个性化和减少两极分化至关重要。