In this work, we examine how fact-checkers prioritize which claims to fact-check and what tools may assist them in their efforts. Through a series of interviews with 23 professional fact-checkers from around the world, we validate that harm assessment is a central component of how fact-checkers triage their work. We also clarify the processes behind fact-checking prioritization, finding that they are typically ad hoc, and gather suggestions for tools that could help with these processes. To address the needs articulated by fact-checkers, we present a structured framework of questions to help fact-checkers negotiate the priority of claims through assessing potential harms. Our FABLE Framework of Misinformation Harms incorporates five dimensions of magnitude -- (social) Fragmentation, Actionability, Believability, Likelihood of spread, and Exploitativeness -- that can help determine the potential urgency of a specific message or claim when considering misinformation as harm. The result is a practical and conceptual tool to support fact-checkers and others as they make strategic decisions to prioritize their efforts. We conclude with a discussion of computational approaches to support structured prioritization, as well as applications beyond fact-checking to content moderation and curation.
翻译:在本研究中,我们探讨了事实核查员如何优先选择要核查的声明,以及哪些工具可辅助其工作。通过对来自全球的23名专业事实核查员进行系列访谈,我们验证了危害评估是事实核查员工作分流中的核心要素。同时厘清了事实核查优先级决策的流程(发现其通常具有临时性特征),并收集了关于辅助工具的建议。为响应事实核查员提出的需求,我们构建了结构化的问题框架,通过评估潜在危害来协助事实核查员协商声明优先级。本研究的FABLE错误信息危害框架包含五个维度指标——(社会)分裂性、可行动性、可信度、传播可能性与利用性——当将错误信息视为危害时,该框架可帮助判断特定信息或声明的潜在紧迫性。最终形成了支持事实核查员及其他相关人员战略决策的实用概念工具,以确定工作优先级。我们最后讨论了支撑结构化优先级的计算方法,以及该框架在内容审核与策展等事实核查之外领域的应用。