Balancing efficiency and accuracy is a long-standing problem for deploying deep learning models. The trade-off is even more important for real-time safety-critical systems like autonomous vehicles. In this paper, we propose an effective approach for accelerating transformer-based 3D object detectors by dynamically halting tokens at different layers depending on their contribution to the detection task. Although halting a token is a non-differentiable operation, our method allows for differentiable end-to-end learning by leveraging an equivalent differentiable forward-pass. Furthermore, our framework allows halted tokens to be reused to inform the model's predictions through a straightforward token recycling mechanism. Our method significantly improves the Pareto frontier of efficiency versus accuracy when compared with the existing approaches. By halting tokens and increasing model capacity, we are able to improve the baseline model's performance without increasing the model's latency on the Waymo Open Dataset.

翻译：平衡效率与精度是部署深度学习模型时长期面临的挑战，这一权衡在实时安全关键系统（如自动驾驶）中尤为重要。本文提出一种高效方法，通过根据令牌对检测任务的贡献度，在不同层动态停止令牌，从而加速基于Transformer的3D目标检测器。尽管停止令牌属于不可微操作，但我们的方法通过利用等效的可微前向传播，实现了可微分的端到端学习。此外，框架允许通过轻量级令牌回收机制，将已停止的令牌重新用于辅助模型预测。与现有方法相比，我们的方法显著提升了效率与精度的帕累托前沿。通过在Waymo开放数据集上停止令牌并扩展模型容量，我们在不增加延迟的情况下提升了基线模型的性能。