Applying deep neural networks to 3D point cloud processing has attracted increasing attention due to its advanced performance in many areas, such as AR/VR, autonomous driving, and robotics. However, as neural network models and 3D point clouds expand in size, it becomes a crucial challenge to reduce the computational and memory overhead to meet latency and energy constraints in real-world applications. Although existing approaches have proposed to reduce both computational cost and memory footprint, most of them only address the spatial redundancy in inputs, i.e. removing the redundancy of background points in 3D data. In this paper, we propose a novel post-training weight pruning scheme for 3D object detection that is (1) orthogonal to all existing point cloud sparsifying methods, which determines redundant parameters in the pretrained model that lead to minimal distortion in both locality and confidence (detection distortion); and (2) a universal plug-and-play pruning framework that works with arbitrary 3D detection model. This framework aims to minimize detection distortion of network output to maximally maintain detection precision, by identifying layer-wise sparsity based on second-order Taylor approximation of the distortion. Albeit utilizing second-order information, we introduced a lightweight scheme to efficiently acquire Hessian information, and subsequently perform dynamic programming to solve the layer-wise sparsity. Extensive experiments on KITTI, Nuscenes and ONCE datasets demonstrate that our approach is able to maintain and even boost the detection precision on pruned model under noticeable computation reduction (FLOPs). Noticeably, we achieve over 3.89x, 3.72x FLOPs reduction on CenterPoint and PVRCNN model, respectively, without mAP decrease, significantly improving the state-of-the-art.

翻译：将深度神经网络应用于三维点云处理因其在增强现实/虚拟现实、自动驾驶和机器人等众多领域展现出的卓越性能而日益受到关注。然而，随着神经网络模型和三维点云数据规模的不断扩大，降低计算和内存开销以满足实际应用中的延迟和能耗约束已成为一项关键挑战。尽管现有方法已提出多种方案以同时降低计算成本和内存占用，但其中大多数仅关注输入数据中的空间冗余，即去除三维数据中背景点的冗余。本文提出了一种新颖的面向三维目标检测的训练后权重剪枝方案，其特点在于：(1) 与所有现有点云稀疏化方法正交，该方法旨在识别预训练模型中导致局部性和置信度（检测失真）最小失真的冗余参数；(2) 一个通用的即插即用剪枝框架，可与任意三维检测模型兼容。该框架基于失真二阶泰勒展开确定逐层稀疏度，旨在最小化网络输出的检测失真，从而最大限度地保持检测精度。尽管利用了二阶信息，我们引入了一种轻量级方案来高效获取海森矩阵信息，并随后通过动态规划求解逐层稀疏度。在KITTI、Nuscenes和ONCE数据集上的大量实验表明，我们的方法能够在显著降低计算量（浮点运算次数）的情况下，维持甚至提升剪枝后模型的检测精度。值得注意的是，我们在CenterPoint和PVRCNN模型上分别实现了超过3.89倍和3.72倍的浮点运算次数减少，且平均精度均值（mAP）未下降，显著提升了现有技术水平。