We present YOLOBench, a benchmark comprised of 550+ YOLO-based object detection models on 4 different datasets and 4 different embedded hardware platforms (x86 CPU, ARM CPU, Nvidia GPU, NPU). We collect accuracy and latency numbers for a variety of YOLO-based one-stage detectors at different model scales by performing a fair, controlled comparison of these detectors with a fixed training environment (code and training hyperparameters). Pareto-optimality analysis of the collected data reveals that, if modern detection heads and training techniques are incorporated into the learning process, multiple architectures of the YOLO series achieve a good accuracy-latency trade-off, including older models like YOLOv3 and YOLOv4. We also evaluate training-free accuracy estimators used in neural architecture search on YOLOBench and demonstrate that, while most state-of-the-art zero-cost accuracy estimators are outperformed by a simple baseline like MAC count, some of them can be effectively used to predict Pareto-optimal detection models. We showcase that by using a zero-cost proxy to identify a YOLO architecture competitive against a state-of-the-art YOLOv8 model on a Raspberry Pi 4 CPU. The code and data are available at https://github.com/Deeplite/deeplite-torch-zoo

翻译：我们提出了YOLOBench基准，它包含550多个基于YOLO的目标检测模型，覆盖4个不同数据集和4种不同嵌入式硬件平台（x86 CPU、ARM CPU、Nvidia GPU、NPU）。我们在固定的训练环境（代码和训练超参数）下对多种不同模型规模的YOLO单阶段检测器进行公平且受控的比较，从而收集其精度和时延数据。对收集数据的帕累托最优分析表明，若将现代检测头与训练技术融入学习过程，则YOLO系列的多种架构（包括YOLOv3和YOLOv4等早期模型）均能实现良好的精度-时延权衡。我们还评估了YOLOBench上用于神经架构搜索的无训练精度估计器，并证明：虽然大多数最先进的零成本精度估计器的性能已被MAC计数等简单基线所超越，但其中部分估计器可有效用于预测帕累托最优检测模型。我们展示了一个实例，即通过使用零成本代理在树莓派4 CPU上识别出与最先进的YOLOv8模型相竞争的YOLO架构。相关代码与数据可在https://github.com/Deeplite/deeplite-torch-zoo获取。