Removing clutter from scenes is essential in many applications, ranging from privacy-concerned content filtering to data augmentation. In this work, we present an automatic system that removes clutter from 3D scenes and inpaints with coherent geometry and texture. We propose techniques for its two key components: 3D segmentation from shared properties and 3D inpainting, both of which are important problems. The definition of 3D scene clutter (frequently-moving objects) is not well captured by commonly-studied object categories in computer vision. To tackle the lack of well-defined clutter annotations, we group noisy fine-grained labels, leverage virtual rendering, and impose an instance-level area-sensitive loss. Once clutter is removed, we inpaint geometry and texture in the resulting holes by merging inpainted RGB-D images. This requires novel voting and pruning strategies that guarantee multi-view consistency across individually inpainted images for mesh reconstruction. Experiments on ScanNet and Matterport dataset show that our method outperforms baselines for clutter segmentation and 3D inpainting, both visually and quantitatively.

翻译：从场景中移除杂波对于诸多应用至关重要，涉及隐私敏感内容过滤至数据增强等范围。本研究提出一套自动系统，可移除3D场景中的杂波，并以连贯几何结构与纹理进行修复。我们针对两大核心组件提出关键技术：基于共享属性的3D分割与3D修复，此二者均为重要研究课题。3D场景杂波（频繁移动物体）的定义未被计算机视觉中常见物体类别充分涵盖。为解决杂波标注缺失问题，我们聚合噪声细粒度标签、利用虚拟渲染，并引入实例级面积敏感损失函数。移除杂波后，通过融合修复的RGB-D图像，对生成的孔洞进行几何与纹理修复。这需要新型投票与剪枝策略，确保各独立修复图像的多视图一致性以重建网格。在ScanNet与Matterport数据集上的实验表明，本方法在杂波分割与3D修复任务中，无论视觉质量还是量化指标均优于基线方法。