Recent years have witnessed an interesting phenomenon in which users come together to interrogate potentially harmful algorithmic behaviors they encounter in their everyday lives. Researchers have started to develop theoretical and empirical understandings of these user driven audits, with a hope to harness the power of users in detecting harmful machine behaviors. However, little is known about user participation and their division of labor in these audits, which are essential to support these collective efforts in the future. Through collecting and analyzing 17,984 tweets from four recent cases of user driven audits, we shed light on patterns of user participation and engagement, especially with the top contributors in each case. We also identified the various roles user generated content played in these audits, including hypothesizing, data collection, amplification, contextualization, and escalation. We discuss implications for designing tools to support user driven audits and users who labor to raise awareness of algorithm bias.

翻译：近年来出现了一个有趣的现象：用户们共同审视他们在日常生活中遇到的潜在有害算法行为。研究人员已开始从理论和实证层面理解这些用户驱动的审计行为，期望借助用户力量检测机器的有害行为。然而，关于用户参与及其在这些审计中的劳动分工（这对未来支持此类集体行动至关重要）仍知之甚少。通过收集并分析四个近期用户驱动审计案例中的17984条推文，我们揭示了用户参与和互动的模式，特别是每个案例中主要贡献者的特征。我们还识别了用户生成内容在这些审计中扮演的不同角色，包括提出假设、数据收集、放大传播、情境化解释和逐级上报。最后讨论了设计工具以支持用户驱动审计，以及为提升算法偏见认知而付出劳动的用户的启示。