Semantic Scene Completion (SSC) transforms an image of single-view depth and/or RGB 2D pixels into 3D voxels, each of whose semantic labels are predicted. SSC is a well-known ill-posed problem as the prediction model has to "imagine" what is behind the visible surface, which is usually represented by Truncated Signed Distance Function (TSDF). Due to the sensory imperfection of the depth camera, most existing methods based on the noisy TSDF estimated from depth values suffer from 1) incomplete volumetric predictions and 2) confused semantic labels. To this end, we use the ground-truth 3D voxels to generate a perfect visible surface, called TSDF-CAD, and then train a "cleaner" SSC model. As the model is noise-free, it is expected to focus more on the "imagination" of unseen voxels. Then, we propose to distill the intermediate "cleaner" knowledge into another model with noisy TSDF input. In particular, we use the 3D occupancy feature and the semantic relations of the "cleaner self" to supervise the counterparts of the "noisy self" to respectively address the above two incorrect predictions. Experimental results validate that our method improves the noisy counterparts with 3.1% IoU and 2.2% mIoU for measuring scene completion and SSC, and also achieves new state-of-the-art accuracy on the popular NYU dataset.

翻译：语义场景补全（SSC）将单视角深度和/或RGB二维像素图像转换为三维体素，并预测每个体素的语义标签。由于预测模型需要“想象”可见表面背后的内容，这通常由截断符号距离函数（TSDF）表示，因此SSC是一个典型的不适定问题。受深度相机感知缺陷的影响，大多数基于噪声TSDF（从深度值估计得到）的现有方法存在两个问题：1）不完整的体积预测；2）混淆的语义标签。为此，我们利用真实三维体素生成完美的可见表面，称为TSDF-CAD，并训练一个“更清洁”的SSC模型。由于该模型无噪声，它应更专注于对不可见体素的“想象”。接着，我们提出将中间“更清洁”的知识蒸馏到另一个输入为噪声TSDF的模型中。具体而言，我们利用“更清洁自监督”的三维占用特征和语义关系来监督“噪声自监督”的对应部分，分别解决上述两种错误预测。实验结果表明，我们的方法使噪声模型的场景补全和SSC性能提升3.1%的IoU和2.2%的mIoU，并在流行的NYU数据集上达到新的最先进精度。