The reliability of supervised machine learning systems depends on the accuracy and availability of ground truth labels. However, the process of human annotation, being prone to error, introduces the potential for noisy labels, which can impede the practicality of these systems. While training with noisy labels is a significant consideration, the reliability of test data is also crucial to ascertain the dependability of the results. A common approach to addressing this issue is repeated labeling, where multiple annotators label the same example, and their labels are combined to provide a better estimate of the true label. In this paper, we propose a novel localization algorithm that adapts well-established ground truth estimation methods for object detection and instance segmentation tasks. The key innovation of our method lies in its ability to transform combined localization and classification tasks into classification-only problems, thus enabling the application of techniques such as Expectation-Maximization (EM) or Majority Voting (MJV). Although our main focus is the aggregation of unique ground truth for test data, our algorithm also shows superior performance during training on the TexBiG dataset, surpassing both noisy label training and label aggregation using Weighted Boxes Fusion (WBF). Our experiments indicate that the benefits of repeated labels emerge under specific dataset and annotation configurations. The key factors appear to be (1) dataset complexity, the (2) annotator consistency, and (3) the given annotation budget constraints.

翻译：监督机器学习系统的可靠性依赖于真实标注的准确性和可用性。然而，人工标注过程容易出错，会引入噪声标签，从而妨碍这些系统的实用性。尽管使用噪声标签进行训练是一个重要考量，但测试数据的可靠性对于确定结果的可靠性同样至关重要。解决此问题的一种常见方法是重复标注，即多个标注者标注同一示例，并将他们的标注结果合并，以提供对真实标签的更好估计。在本文中，我们提出了一种新颖的定位算法，该算法将成熟的目标检测和实例分割任务中的真实标注估计方法进行了改进。我们方法的关键创新在于其能够将定位和分类的组合任务转化为纯分类问题，从而能够应用期望最大化（EM）或多数投票（MJV）等技术。虽然我们的主要关注点是聚合测试数据的唯一真实标注，但我们的算法在TexBiG数据集上的训练过程中也展现出优越性能，超越了使用加权框融合（WBF）的噪声标签训练和标签聚合。我们的实验表明，重复标注的优势在特定的数据集和标注配置下才会显现。关键因素包括：（1）数据集复杂度，（2）标注者一致性，以及（3）给定的标注预算约束。