Unsupervised domain adaptation (UDA) in 3D segmentation tasks presents a formidable challenge, primarily stemming from the sparse and unordered nature of point cloud data. Especially for LiDAR point clouds, the domain discrepancy becomes obvious across varying capture scenes, fluctuating weather conditions, and the diverse array of LiDAR devices in use. While previous UDA methodologies have often sought to mitigate this gap by aligning features between source and target domains, this approach falls short when applied to 3D segmentation due to the substantial domain variations. Inspired by the remarkable generalization capabilities exhibited by the vision foundation model, SAM, in the realm of image segmentation, our approach leverages the wealth of general knowledge embedded within SAM to unify feature representations across diverse 3D domains and further solves the 3D domain adaptation problem. Specifically, we harness the corresponding images associated with point clouds to facilitate knowledge transfer and propose an innovative hybrid feature augmentation methodology, which significantly enhances the alignment between the 3D feature space and SAM's feature space, operating at both the scene and instance levels. Our method is evaluated on many widely-recognized datasets and achieves state-of-the-art performance.
翻译:三维分割任务中的无监督域适应(UDA)面临重大挑战,主要源于点云数据的稀疏性和无序性。尤其在激光雷达点云中,不同采集场景、多变天气条件以及各类激光雷达设备的广泛使用,使得域差异变得尤为显著。尽管以往的UDA方法常通过对齐源域和目标域之间的特征来缩小这一差异,但由于三维分割中域变化巨大,此方法效果有限。受视觉基础模型SAM在图像分割领域展示出的卓越泛化能力启发,我们利用SAM蕴含的丰富通用知识来统一不同三维域的特征表示,从而解决三维域适应问题。具体而言,我们利用与点云对应的图像促进知识迁移,并提出一种创新的混合特征增强方法,从场景和实例两个层面显著提升三维特征空间与SAM特征空间的对齐效果。该方法在多个广泛认可的数据集上进行了评估,并达到了最先进的性能。