We are amidst an explosion of artificial intelligence research, particularly around large language models (LLMs). These models have a range of applications across domains like medicine, finance, commonsense knowledge graphs, and crowdsourcing. Investigation into LLMs as part of crowdsourcing workflows remains an under-explored space. The crowdsourcing research community has produced a body of work investigating workflows and methods for managing complex tasks using hybrid human-AI methods. Within crowdsourcing, the role of LLMs can be envisioned as akin to a cog in a larger wheel of workflows. From an empirical standpoint, little is currently understood about how LLMs can improve the effectiveness of crowdsourcing workflows and how such workflows can be evaluated. In this work, we present a vision for exploring this gap from the perspectives of various stakeholders involved in the crowdsourcing paradigm -- the task requesters, crowd workers, platforms, and end-users. We identify junctures in typical crowdsourcing workflows at which the introduction of LLMs can play a beneficial role and propose means to augment existing design patterns for crowd work.
翻译:我们正身处人工智能研究的爆发时代,尤其是大语言模型(LLMs)领域。这些模型在医学、金融、常识知识图谱及众包等跨领域应用中展现出广泛潜力。将LLMs融入众包工作流的研究目前仍是一个待深入探索的领域。众包研究社区已产出了大量关于利用人类-人工智能混合方法管理复杂任务的工作流与方法的研究。在众包体系中,LLMs的角色可被视作大型工作流齿轮中的一个齿轮。从实证角度来看,目前尚未充分理解LLMs如何提升众包工作流的效率,以及如何评估此类工作流。本工作提出从众包范式中各利益相关者(任务发布方、众包工作者、平台及终端用户)的视角探索这一研究空白的愿景。我们识别出典型众包工作流中引入LLMs能发挥优势的关键节点,并提出增强现有众包设计模式的改进方案。