Signed Distance Functions (SDFs) are vital implicit representations to represent high fidelity 3D surfaces. Current methods mainly leverage a neural network to learn an SDF from various supervisions including signed distances, 3D point clouds, or multi-view images. However, due to various reasons including the bias of neural network on low frequency content, 3D unaware sampling, sparsity in point clouds, or low resolutions of images, neural implicit representations still struggle to represent geometries with high frequency components like sharp structures, especially for the ones learned from images or point clouds. To overcome this challenge, we introduce a method to sharpen a low frequency SDF observation by recovering its high frequency components, pursuing a sharper and more complete surface. Our key idea is to learn a mapping from a low frequency observation to a full frequency coverage in a data-driven manner, leading to a prior knowledge of shape consolidation in the frequency domain, dubbed frequency consolidation priors. To better generalize a learned prior to unseen shapes, we introduce to represent frequency components as embeddings and disentangle the embedding of the low frequency component from the embedding of the full frequency component. This disentanglement allows the prior to generalize on an unseen low frequency observation by simply recovering its full frequency embedding through a test-time self-reconstruction. Our evaluations under widely used benchmarks or real scenes show that our method can recover high frequency component and produce more accurate surfaces than the latest methods. The code, data, and pre-trained models are available at \url{https://github.com/chenchao15/FCP}.
翻译:符号距离函数(SDF)是表示高保真三维曲面的关键隐式表示方法。现有方法主要利用神经网络从符号距离、三维点云或多视角图像等多种监督信号中学习SDF。然而,由于神经网络对低频内容的固有偏好、三维空间采样不充分、点云数据稀疏性以及图像分辨率有限等原因,神经隐式表示在刻画具有高频成分(如锐利结构)的几何形状时仍面临挑战,尤其对于从图像或点云学习得到的表示。为克服这一难题,我们提出一种通过恢复高频分量来锐化低频SDF观测值的方法,以获取更锐利、更完整的曲面。我们的核心思想是以数据驱动的方式学习从低频观测到全频覆盖的映射,从而构建频域形状整合的先验知识,称为频率整合先验。为提升学习先验对未见形状的泛化能力,我们提出将频率分量表示为嵌入向量,并将低频分量的嵌入与全频分量的嵌入进行解耦。这种解耦设计使得先验能够通过测试时的自重构过程恢复全频嵌入,从而泛化至未见过的低频观测。在广泛使用的基准数据集和真实场景下的评估表明,我们的方法能有效恢复高频分量,相比最新方法能生成更精确的曲面。代码、数据及预训练模型已公开于 \url{https://github.com/chenchao15/FCP}。