Recent advances in computer vision have made training object detectors more efficient and effective; however, assessing their performance in real-world applications still relies on costly manual annotation. To address this limitation, we develop an automated model evaluation (AutoEval) framework for object detection. We propose Prediction Consistency and Reliability (PCR), which leverages the multiple candidate bounding boxes that conventional detectors generate before non-maximum suppression (NMS). PCR estimates detection performance without ground-truth labels by jointly measuring 1) the spatial consistency between boxes before and after NMS, and 2) the reliability of the retained boxes via the confidence scores of overlapping boxes. For a more realistic and scalable evaluation, we construct a meta-dataset by applying image corruptions of varying severity. Experimental results demonstrate that PCR yields more accurate performance estimates than existing AutoEval methods, and the proposed meta-dataset covers a wider range of detection performance. The code is available at https://github.com/YonseiML/autoeval-det.

翻译：近年来计算机视觉领域的进展使得目标检测器的训练更加高效；然而，在实际应用中评估其性能仍依赖于昂贵的人工标注。为克服这一局限，我们针对目标检测任务开发了一种自动化模型评估（AutoEval）框架。我们提出了预测一致性与可靠性（PCR）指标，该指标利用传统检测器在非极大值抑制（NMS）前生成的多个候选边界框。PCR通过联合度量以下两个要素，在无需真实标注的情况下估计检测性能：1）NMS前后边界框的空间一致性；2）通过重叠框的置信度分数评估保留框的可靠性。为实现更贴近实际且可扩展的评估，我们通过施加不同严重程度的图像退化构建了一个元数据集。实验结果表明，与现有AutoEval方法相比，PCR能产生更准确的性能估计，且所提出的元数据集覆盖了更广泛的检测性能范围。代码公开于：https://github.com/YonseiML/autoeval-det。