Moving deep learning models from the laboratory setting to the open world entails preparing them to handle unforeseen conditions. In several applications the occurrence of novel classes during deployment poses a significant threat, thus it is crucial to effectively detect them. Ideally, this skill should be used when needed without requiring any further computational training effort at every new task. Out-of-distribution detection has attracted significant attention in the last years, however the majority of the studies deal with 2D images ignoring the inherent 3D nature of the real-world and often confusing between domain and semantic novelty. In this work, we focus on the latter, considering the objects geometric structure captured by 3D point clouds regardless of the specific domain. We advance the field by introducing OpenPatch that builds on a large pre-trained model and simply extracts from its intermediate features a set of patch representations that describe each known class. For any new sample, we obtain a novelty score by evaluating whether it can be recomposed mainly by patches of a single known class or rather via the contribution of multiple classes. We present an extensive experimental evaluation of our approach for the task of semantic novelty detection on real-world point cloud samples when the reference known data are synthetic. We demonstrate that OpenPatch excels in both the full and few-shot known sample scenarios, showcasing its robustness across varying pre-training objectives and network backbones. The inherent training-free nature of our method allows for its immediate application to a wide array of real-world tasks, offering a compelling advantage over approaches that need expensive retraining efforts.

翻译：将深度学习模型从实验室环境迁移至开放世界，需使其具备应对未知条件的能力。在许多应用中，部署期间出现新类别会构成重大威胁，因此高效检测这些新类别至关重要。理想情况下，这种能力应在需要时自主启用，且无需为每个新任务进行额外的计算训练。近年来，异常检测虽已引起广泛关注，但多数研究聚焦于二维图像，忽视了现实世界固有的三维属性，且常混淆领域偏移与语义新颖性。本文聚焦后一问题，关注三维点云所捕获的物体几何结构，不受特定领域限制。我们提出OpenPatch方法，该方法基于大规模预训练模型，仅通过提取其中间特征构建一组描述每个已知类别的补丁表示。对于新样本，通过评估该样本是否能由单一已知类别的补丁主导重构，或需依赖多个类别的贡献，从而获取新颖性评分。我们针对语义新颖性检测任务开展了广泛实验评估，在参考已知数据为合成数据的情况下，对真实点云样本进行测试。实验表明，OpenPatch在全样本与少样本已知场景中均表现优异，且在不同预训练目标与网络主干下均展现出鲁棒性。该方法固有的免训练特性使其可即时应用于实际任务，相比需要昂贵重训练的现有方案具有显著优势。