Point cloud analysis (such as 3D segmentation and detection) is a challenging task, because of not only the irregular geometries of many millions of unordered points, but also the great variations caused by depth, viewpoint, occlusion, etc. Current studies put much focus on the adaption of neural networks to the complex geometries of point clouds, but are blind to a fundamental question: how to learn an appropriate point embedding space that is aware of both discriminative semantics and challenging variations? As a response, we propose a clustering based supervised learning scheme for point cloud analysis. Unlike current de-facto, scene-wise training paradigm, our algorithm conducts within-class clustering on the point embedding space for automatically discovering subclass patterns which are latent yet representative across scenes. The mined patterns are, in turn, used to repaint the embedding space, so as to respect the underlying distribution of the entire training dataset and improve the robustness to the variations. Our algorithm is principled and readily pluggable to modern point cloud segmentation networks during training, without extra overhead during testing. With various 3D network architectures (i.e., voxel-based, point-based, Transformer-based, automatically searched), our algorithm shows notable improvements on famous point cloud segmentation datasets (i.e.,2.0-2.6% on single-scan and 2.0-2.2% multi-scan of SemanticKITTI, 1.8-1.9% on S3DIS, in terms of mIoU). Our algorithm also demonstrates utility in 3D detection, showing 2.0-3.4% mAP gains on KITTI.

翻译：点云分析（如三维分割与检测）是一项具有挑战性的任务，不仅因为数百万无序点的不规则几何结构，还因深度、视角、遮挡等因素导致的巨大变化。现有研究多聚焦于神经网络对点云复杂几何结构的适应，却忽视了一个根本性问题：如何学习一个既感知判别性语义又应对复杂变化的恰当点嵌入空间？为此，我们提出一种基于聚类的监督学习方案用于点云分析。不同于当前事实上的逐场景训练范式，我们的算法在点嵌入空间内执行类内聚类，以自动发现跨场景中潜在且具代表性的子类模式。挖掘到的模式反过来用于重塑嵌入空间，从而尊重整个训练数据集的底层分布，并增强对变化的鲁棒性。该算法原理清晰，在训练阶段可即插即用于现代点云分割网络，且测试阶段不引入额外开销。在多种三维网络架构（如体素型、点型、Transformer型、自动搜索型）上，我们的算法在著名点云分割数据集上均取得显著提升（SemanticKITTI单扫描mIoU提升2.0-2.6%，多扫描提升2.0-2.2%；S3DIS提升1.8-1.9%）。该算法在三维检测任务中同样有效，在KITTI数据集上获得2.0-3.4%的mAP增益。