Efficient inference for object detection networks is a major challenge on edge devices. Post-Training Quantization (PTQ), which transforms a full-precision model into low bit-width directly, is an effective and convenient approach to reduce model inference complexity. But it suffers severe accuracy drop when applied to complex tasks such as object detection. PTQ optimizes the quantization parameters by different metrics to minimize the perturbation of quantization. The p-norm distance of feature maps before and after quantization, Lp, is widely used as the metric to evaluate perturbation. For the specialty of object detection network, we observe that the parameter p in Lp metric will significantly influence its quantization performance. We indicate that using a fixed hyper-parameter p does not achieve optimal quantization performance. To mitigate this problem, we propose a framework, DetPTQ, to assign different p values for quantizing different layers using an Object Detection Output Loss (ODOL), which represents the task loss of object detection. DetPTQ employs the ODOL-based adaptive Lp metric to select the optimal quantization parameters. Experiments show that our DetPTQ outperforms the state-of-the-art PTQ methods by a significant margin on both 2D and 3D object detectors. For example, we achieve 31.1/31.7(quantization/full-precision) mAP on RetinaNet-ResNet18 with 4-bit weight and 4-bit activation.

翻译：边缘设备上目标检测网络的高效推理是一项重大挑战。后训练量化（PTQ）能直接将全精度模型转化为低位宽模型，是降低模型推理复杂度的一种有效且便捷的方法。然而，当应用于目标检测等复杂任务时，该方法的精度会严重下降。PTQ通过不同度量标准优化量化参数，以最小化量化带来的扰动。特征图量化前后的p范数距离（即Lp度量）被广泛用作评估扰动的指标。针对目标检测网络的特异性，我们观察到Lp度量中的参数p会显著影响其量化性能。研究表明，使用固定的超参数p无法达到最优的量化性能。为解决这一问题，我们提出DetPTQ框架，通过目标检测输出损失（ODOL）为不同层的量化分配不同的p值，该损失表征了目标检测的任务损失。DetPTQ采用基于ODOL的自适应Lp度量来选择最优量化参数。实验表明，我们的DetPTQ在2D和3D目标检测器上均显著优于现有最先进的PTQ方法。例如，在RetinaNet-ResNet18模型上采用4位权重和4位激活量化时，我们实现了31.1/31.7（量化/全精度）的mAP。