Current 3D segmentation methods heavily rely on large-scale point-cloud datasets, which are notoriously laborious to annotate. Few attempts have been made to circumvent the need for dense per-point annotations. In this work, we look at weakly-supervised 3D semantic instance segmentation. The key idea is to leverage 3D bounding box labels which are easier and faster to annotate. Indeed, we show that it is possible to train dense segmentation models using only bounding box labels. At the core of our method, \name{}, lies a deep model, inspired by classical Hough voting, that directly votes for bounding box parameters, and a clustering method specifically tailored to bounding box votes. This goes beyond commonly used center votes, which would not fully exploit the bounding box annotations. On ScanNet test, our weakly supervised model attains leading performance among other weakly supervised approaches (+18 mAP@50). Remarkably, it also achieves 97% of the mAP@50 score of current fully supervised models. To further illustrate the practicality of our work, we train Box2Mask on the recently released ARKitScenes dataset which is annotated with 3D bounding boxes only, and show, for the first time, compelling 3D instance segmentation masks.

翻译：当前的三维分割方法高度依赖大规模点云数据集，而这类数据的逐点标注极为耗时费力。目前鲜有研究尝试规避密集逐点标注的需求。本文聚焦于弱监督三维语义实例分割问题，核心思想是利用标注更简便快捷的三维边界框标签。我们证明，仅凭边界框标签即可训练出高密度分割模型。本方法的核心名为\name{}，其基础架构受经典霍夫投票启发：深度网络直接投票生成边界框参数，并配合专为边界框投票设计的聚类方法。这一设计超越了仅利用中心点投票的常规做法——后者无法充分挖掘边界框标注信息。在ScanNet测试集上，我们的弱监督模型在同类方法中取得领先性能（mAP@50提升18%），尤为突出的是，其mAP@50得分达到了当前全监督模型的97%。为进一步验证实用性，我们在仅含三维边界框标注的ARKitScenes数据集上训练Box2Mask，首次成功获得了令人信服的三维实例分割掩膜。