The task of Novel Class Discovery (NCD) in semantic segmentation entails training a model able to accurately segment unlabelled (novel) classes, relying on the available supervision from annotated (base) classes. Although extensively investigated in 2D image data, the extension of the NCD task to the domain of 3D point clouds represents a pioneering effort, characterized by assumptions and challenges that are not present in the 2D case. This paper represents an advancement in the analysis of point cloud data in four directions. Firstly, it introduces the novel task of NCD for point cloud semantic segmentation. Secondly, it demonstrates that directly transposing the only existing NCD method for 2D image semantic segmentation to 3D data yields suboptimal results. Thirdly, a new NCD approach based on online clustering, uncertainty estimation, and semantic distillation is presented. Lastly, a novel evaluation protocol is proposed to rigorously assess the performance of NCD in point cloud semantic segmentation. Through comprehensive evaluations on the SemanticKITTI, SemanticPOSS, and S3DIS datasets, the paper demonstrates substantial superiority of the proposed method over the considered baselines.
翻译:语义分割中的新类别发现任务旨在训练一个模型,使其能够基于已标注基础类别的可用监督信息,准确分割未标注的新类别。尽管该任务在二维图像数据中已得到广泛研究,但将其扩展至三维点云领域仍属开创性工作,其面临的假设与挑战在二维场景中并不存在。本文在点云数据分析方面取得了四个方向的进展。首先,提出了点云语义分割的新类别发现任务。其次,证明了直接将现有唯一的二维图像语义分割NCD方法移植到三维数据会导致次优结果。第三,提出了一种基于在线聚类、不确定性估计和语义蒸馏的新型NCD方法。最后,设计了一种新颖的评估协议以严格评估点云语义分割中NCD的性能。通过在SemanticKITTI、SemanticPOSS和S3DIS数据集上的综合评估,本文证明了所提方法相较于现有基线具有显著优越性。