This paper presents a robust fine-tuning method designed for pre-trained 3D point cloud models, to enhance feature robustness in downstream fine-tuned models. We highlight the limitations of current fine-tuning methods and the challenges of learning robust models. The proposed method, named Weight-Space Ensembles for Fine-Tuning then Linear Probing (WiSE-FT-LP), integrates the original pre-training and fine-tuning models through weight space integration followed by Linear Probing. This approach significantly enhances the performance of downstream fine-tuned models under distribution shifts, improving feature robustness while maintaining high performance on the target distribution. We apply this robust fine-tuning method to mainstream 3D point cloud pre-trained models and evaluate the quality of model parameters and the degradation of downstream task performance. Experimental results demonstrate the effectiveness of WiSE-FT-LP in enhancing model robustness, effectively balancing downstream task performance and model feature robustness without altering the model structures.

翻译：本文提出了一种针对预训练三维点云模型的鲁棒微调方法，旨在增强下游微调模型的特征鲁棒性。我们揭示了当前微调方法的局限性以及学习鲁棒模型面临的挑战。所提出的方法——权重空间集成微调与线性探测（WiSE-FT-LP）——通过权重空间集成并随后进行线性探测，将原始预训练模型与微调模型进行整合。该方法显著提升了分布偏移下下游微调模型的性能，在保持目标分布高性能的同时增强特征鲁棒性。我们将此鲁棒微调方法应用于主流三维点云预训练模型，并评估了模型参数质量及下游任务性能退化情况。实验结果表明，WiSE-FT-LP在增强模型鲁棒性方面具有有效性，能够在不改变模型结构的前提下，有效平衡下游任务性能与模型特征鲁棒性。