A fundamental limitation of object detectors is that they suffer from "spatial bias", and in particular perform less satisfactorily when detecting objects near image borders. For a long time, there has been a lack of effective ways to measure and identify spatial bias, and little is known about where it comes from and what degree it is. To this end, we present a new zone evaluation protocol, extending from the traditional evaluation to a more generalized one, which measures the detection performance over zones, yielding a series of Zone Precisions (ZPs). For the first time, we provide numerical results, showing that the object detectors perform quite unevenly across the zones. Surprisingly, the detector's performance in the 96\% border zone of the image does not reach the AP value (Average Precision, commonly regarded as the average detection performance in the entire image zone). To better understand spatial bias, a series of heuristic experiments are conducted. Our investigation excludes two intuitive conjectures about spatial bias that the object scale and the absolute positions of objects barely influence the spatial bias. We find that the key lies in the human-imperceptible divergence in data patterns between objects in different zones, thus eventually forming a visible performance gap between the zones. With these findings, we finally discuss a future direction for object detection, namely, spatial disequilibrium problem, aiming at pursuing a balanced detection ability over the entire image zone. By broadly evaluating 10 popular object detectors and 5 detection datasets, we shed light on the spatial bias of object detectors. We hope this work could raise a focus on detection robustness. The source codes, evaluation protocols, and tutorials are publicly available at \url{https://github.com/Zzh-tju/ZoneEval}.
翻译:目标检测器的一个基本限制是其存在“空间偏差”,特别是在检测图像边界附近的物体时表现较差。长期以来,缺乏有效的方法来测量和识别空间偏差,对其来源和程度知之甚少。为此,我们提出了一种新的区域评估协议,将传统评估扩展为更通用的形式,通过测量不同区域的检测性能,得到一系列区域精度(Zone Precisions, ZPs)。我们首次提供了数值结果,表明目标检测器在不同区域的性能差异显著。令人惊讶的是,检测器在图像96%边界区域的性能并未达到AP值(平均精度,通常被视为整个图像区域的平均检测性能)。为了更好地理解空间偏差,我们进行了一系列启发式实验。我们的研究排除了关于空间偏差的两个直观猜想,即物体尺度和物体绝对位置对空间偏差几乎没有影响。我们发现关键在于不同区域中物体数据模式存在人类难以察觉的差异,从而最终形成了区域间可见的性能差距。基于这些发现,我们最终讨论了目标检测的一个未来方向,即空间失衡问题,旨在追求整个图像区域的均衡检测能力。通过对10种流行目标检测器和5个检测数据集的广泛评估,我们揭示了目标检测器的空间偏差。我们希望这项工作能引起对检测鲁棒性的关注。源代码、评估协议和教程已在 \url{https://github.com/Zzh-tju/ZoneEval} 公开提供。