A signed distance function (SDF) parametrized by an MLP is a common ingredient of neural surface reconstruction. We build on the successful recent method NeuS to extend it by three new components. The first component is to borrow the tri-plane representation from EG3D and represent signed distance fields as a mixture of tri-planes and MLPs instead of representing it with MLPs only. Using tri-planes leads to a more expressive data structure but will also introduce noise in the reconstructed surface. The second component is to use a new type of positional encoding with learnable weights to combat noise in the reconstruction process. We divide the features in the tri-plane into multiple frequency scales and modulate them with sin and cos functions of different frequencies. The third component is to use learnable convolution operations on the tri-plane features using self-attention convolution to produce features with different frequency bands. The experiments show that PET-NeuS achieves high-fidelity surface reconstruction on standard datasets. Following previous work and using the Chamfer metric as the most important way to measure surface reconstruction quality, we are able to improve upon the NeuS baseline by 57% on Nerf-synthetic (0.84 compared to 1.97) and by 15.5% on DTU (0.71 compared to 0.84). The qualitative evaluation reveals how our method can better control the interference of high-frequency noise. Code available at \url{https://github.com/yiqun-wang/PET-NeuS}.
翻译:符号距离函数(SDF)通常由多层感知机(MLP)参数化,是神经曲面重建的常见组成部分。我们基于近期成功的NeuS方法进行扩展,新增三个组件。第一个组件借鉴EG3D中的三平面表示,将符号距离场表示为三平面与MLP的混合形式,而非仅用MLP表示。采用三平面可形成更具表达力的数据结构,但也会在重建曲面中引入噪声。第二个组件采用具有可学习权重的新型位置编码方法,以抑制重建过程中的噪声。我们将三平面中的特征划分为多个频率尺度,并使用不同频率的正弦与余弦函数对其进行调制。第三个组件通过自注意力卷积对三平面特征执行可学习卷积运算,生成不同频带特征。实验表明,PET-NeuS在标准数据集上实现了高保真曲面重建。沿用先前工作以倒角距离作为曲面重建质量的核心评估指标,我们在Nerf-synthetic数据集上相比NeuS基线提升57%(0.84对比1.97),在DTU数据集上提升15.5%(0.71对比0.84)。定性评估揭示了我们的方法如何更有效地控制高频噪声干扰。代码开源地址:\url{https://github.com/yiqun-wang/PET-NeuS}。