In recent years, zero-shot learning has attracted the focus of many researchers, due to its flexibility and generality. Many approaches have been proposed to achieve the zero-shot classification of the point clouds for 3D object understanding, following the schema of CLIP. However, in the real world, the point clouds could be extremely sparse, dramatically limiting the effectiveness of the 3D point cloud encoders, and resulting in the misalignment of point cloud features and text embeddings. To the point cloud encoders to fit the extremely sparse point clouds without re-running the pre-training procedure which could be time-consuming and expensive, in this work, we propose an unsupervised model adaptation approach to enhance the point cloud encoder for the extremely sparse point clouds. We propose a novel fused-cross attention layer that expands the pre-trained self-attention layer with additional learnable tokens and attention blocks, which effectively modifies the point cloud features while maintaining the alignment between point cloud features and text embeddings. We also propose a complementary learning-based self-distillation schema that encourages the modified features to be pulled apart from the irrelevant text embeddings without overfitting the feature space to the observed text embeddings. Extensive experiments demonstrate that the proposed approach effectively increases the zero-shot capability on extremely sparse point clouds, and overwhelms other state-of-the-art model adaptation approaches.

翻译：近年来，零样本学习因其灵活性和通用性而吸引了众多研究者的关注。基于CLIP框架，已有多种方法实现了用于三维物体理解的点云零样本分类。然而，在现实场景中，点云可能极度稀疏，这极大地限制了三维点云编码器的有效性，导致点云特征与文本嵌入之间的对齐失效。为避免重新运行耗时且昂贵的预训练过程，使点云编码器适应极端稀疏点云，本文提出一种无监督模型自适应方法以增强点云编码器。我们设计了新型融合交叉注意力层，通过引入额外可学习令牌和注意力块扩展预训练的自注意力层，在保持点云特征与文本嵌入对齐的同时有效修正点云特征。此外，我们提出基于互补学习的自蒸馏框架，在避免特征空间对已观测文本嵌入过拟合的前提下，促使修正特征与无关文本嵌入分离。大量实验表明，所提方法能有效提升极端稀疏点云的零样本分类能力，并超越当前最先进的模型自适应方法。