Recently, pre-trained point cloud models have found extensive applications in downstream tasks like object classification. However, these tasks often require {full fine-tuning} of models and lead to storage-intensive procedures, thus limiting the real applications of pre-trained models. Inspired by the great success of visual prompt tuning (VPT) in vision, we attempt to explore prompt tuning, which serves as an efficient alternative to full fine-tuning for large-scale models, to point cloud pre-trained models to reduce storage costs. However, it is non-trivial to apply the traditional static VPT to point clouds, owing to the distribution diversity of point cloud data. For instance, the scanned point clouds exhibit various types of missing or noisy points. To address this issue, we propose an Instance-aware Dynamic Prompt Tuning (IDPT) for point cloud pre-trained models, which utilizes a prompt module to perceive the semantic prior features of each instance. This semantic prior facilitates the learning of unique prompts for each instance, thus enabling downstream tasks to robustly adapt to pre-trained point cloud models. Notably, extensive experiments conducted on downstream tasks demonstrate that IDPT outperforms full fine-tuning in most tasks with a mere 7\% of the trainable parameters, thus significantly reducing the storage pressure. Code is available at \url{https://github.com/zyh16143998882/IDPT}.
翻译:最近,预训练点云模型在下游任务(如目标分类)中得到了广泛应用。然而,这些任务通常需要对模型进行全量微调,导致存储密集型过程,从而限制了预训练模型的实际应用。受视觉提示调优(VPT)在视觉领域巨大成功的启发,我们尝试探索提示调优——作为大规模模型全量微调的有效替代方案——应用于点云预训练模型,以降低存储成本。但将传统静态VPT直接应用于点云并非易事,因为点云数据的分布具有多样性。例如,扫描点云存在各类缺失或噪声点。为解决这一问题,我们提出面向实例感知的动态提示调优(IDPT)方法,用于点云预训练模型。该方法利用提示模块感知每个实例的语义先验特征,这种语义先验有助于为每个实例学习独特的提示,从而使下游任务能够鲁棒地适应预训练点云模型。值得注意的是,在下游任务上开展的大量实验表明,IDPT在仅使用7%可训练参数的情况下,在大多数任务上超越了全量微调,从而显著降低了存储压力。代码可从\url{https://github.com/zyh16143998882/IDPT}获取。