Recent methods for neural surface representation and rendering, for example NeuS, have demonstrated the remarkably high-quality reconstruction of static scenes. However, the training of NeuS takes an extremely long time (8 hours), which makes it almost impossible to apply them to dynamic scenes with thousands of frames. We propose a fast neural surface reconstruction approach, called NeuS2, which achieves two orders of magnitude improvement in terms of acceleration without compromising reconstruction quality. To accelerate the training process, we parameterize a neural surface representation by multi-resolution hash encodings and present a novel lightweight calculation of second-order derivatives tailored to our networks to leverage CUDA parallelism, achieving a factor two speed up. To further stabilize and expedite training, a progressive learning strategy is proposed to optimize multi-resolution hash encodings from coarse to fine. We extend our method for fast training of dynamic scenes, with a proposed incremental training strategy and a novel global transformation prediction component, which allow our method to handle challenging long sequences with large movements and deformations. Our experiments on various datasets demonstrate that NeuS2 significantly outperforms the state-of-the-arts in both surface reconstruction accuracy and training speed for both static and dynamic scenes. The code is available at our website: https://vcai.mpi-inf.mpg.de/projects/NeuS2/ .
翻译:近期针对神经表面表示与渲染的方法(如NeuS)已展现出静态场景下极高质量的重建效果。然而,NeuS的训练耗时极长(8小时),这使其几乎无法应用于包含数千帧的动态场景。我们提出了一种名为NeuS2的快速神经表面重建方法,在不牺牲重建质量的前提下,实现了两个数量级的加速提升。为加速训练过程,我们采用多分辨率哈希编码对神经表面表示进行参数化,并提出一种针对我们网络的新型轻量级二阶导数计算方法,充分利用CUDA并行性实现两倍加速。为进一步稳定并加速训练,我们提出渐进式学习策略,从粗到细优化多分辨率哈希编码。我们将该方法扩展至动态场景的快速训练:通过引入增量式训练策略与新型全局变换预测模块,使我们的方法能够处理包含大幅运动与形变的挑战性长序列。在多个数据集上的实验表明,NeuS2在静态与动态场景的表面重建精度与训练速度上均显著超越现有最优方法。代码已发布于官方网站:https://vcai.mpi-inf.mpg.de/projects/NeuS2/。