High-quality labeled data is essential for training robust machine learning models, yet obtaining annotations at scale remains expensive. AI-assisted annotation has therefore become standard in large-scale labeling workflows. However, in tasks where model predictions carry two independent components, a class label and spatial boundaries, a model may classify an object with high confidence while mislocalizing it. Existing AI-assisted workflows offer annotators no signal about where spatial errors are most likely. Without such guidance, humans may systematically underinspect subtly misplaced boxes. We address this by studying the effect of visualizing spatial uncertainty via a purpose-built interface. In a controlled study with 120 participants, those receiving uncertainty cues achieve higher label quality while being faster overall. A box-level analysis confirms that the cues redirect annotator effort toward high-uncertainty predictions and away from well-localized boxes. These findings establish localization uncertainty as a lever to improve human-in-the-loop annotation. Code is available at https://mos-ks.github.io/MUHA/.
翻译:高质量标注数据对于训练稳健的机器学习模型至关重要,然而大规模获取标注的成本仍然高昂。因此,人工智能辅助标注已成为大规模标注流程中的标准做法。然而,在模型预测包含两个独立组成部分(类别标签和空间边界)的任务中,模型可能以高置信度对物体进行分类,却对其定位错误。现有的人工智能辅助工作流程未能向标注人员提供空间错误最可能发生位置的信号。缺乏此类引导,人类可能系统性地忽略那些轻微错位的边界框。我们通过研究利用专用界面可视化空间不确定性的效果来解决这一问题。在一项包含120名参与者的对照研究中,接收不确定性线索的参与者在整体速度更快的同时,实现了更高的标注质量。逐框分析证实,这些线索将标注人员的注意力重新导向高不确定性的预测,而远离定位良好的边界框。这些发现确立了定位不确定性作为改进人在环标注的一个有效杠杆。代码可在https://mos-ks.github.io/MUHA/获取。