Crowdsourcing platforms have transformed distributed problem-solving, yet quality control remains a persistent challenge. Traditional quality control measures, such as prescreening workers and refining instructions, often focus solely on optimizing economic output. This paper explores just-in-time AI interventions to enhance both labeling quality and domain-specific knowledge among crowdworkers. We introduce LabelAId, an advanced inference model combining Programmatic Weak Supervision (PWS) with FT-Transformers to infer label correctness based on user behavior and domain knowledge. Our technical evaluation shows that our LabelAId pipeline consistently outperforms state-of-the-art ML baselines, improving mistake inference accuracy by 36.7% with 50 downstream samples. We then implemented LabelAId into Project Sidewalk, an open-source crowdsourcing platform for urban accessibility. A between-subjects study with 34 participants demonstrates that LabelAId significantly enhances label precision without compromising efficiency while also increasing labeler confidence. We discuss LabelAId's success factors, limitations, and its generalizability to other crowdsourced science domains.

翻译：众包平台改变了分布式问题解决模式，但质量控制仍是持续挑战。传统的质量控制手段（如预筛选工作者和优化任务说明）往往仅聚焦于经济产出最大化。本文探索利用即时AI干预来提升众包工作者的标注质量与领域专业知识。我们提出LabelAId——一种结合程序化弱监督（PWS）与FT-Transformer的高级推理模型，通过分析用户行为与领域知识推断标注正确性。技术评估表明，LabelAId流水线持续优于当前最优的机器学习基线，在仅使用50个下游样本的情况下，将错误推理准确率提升36.7%。随后我们将LabelAId部署至城市可访问性开源众包平台Project Sidewalk。一项包含34名参与者的组间实验证实：LabelAId在保持标注效率的同时显著提升标注精度，并增强标注者信心。我们讨论了LabelAId的成功要素、局限性及其在其他众包科学领域的可推广性。