Recently, many works have been proposed to utilize the neural radiance field for novel view synthesis of human performers. However, most of these methods require hours of training, making them difficult for practical use. To address this challenging problem, we propose IntrinsicNGP, which can train from scratch and achieve high-fidelity results in few minutes with videos of a human performer. To achieve this target, we introduce a continuous and optimizable intrinsic coordinate rather than the original explicit Euclidean coordinate in the hash encoding module of instant-NGP. With this novel intrinsic coordinate, IntrinsicNGP can aggregate inter-frame information for dynamic objects with the help of proxy geometry shapes. Moreover, the results trained with the given rough geometry shapes can be further refined with an optimizable offset field based on the intrinsic coordinate.Extensive experimental results on several datasets demonstrate the effectiveness and efficiency of IntrinsicNGP. We also illustrate our approach's ability to edit the shape of reconstructed subjects.
翻译:近期,大量研究工作致力于利用神经辐射场实现人体表演者的新视角合成。然而,这些方法大多需要数小时的训练时间,使其难以实际应用。针对这一难题,我们提出IntrinsicNGP,该方法能从零开始训练,并在数分钟内通过人体表演者视频获得高保真结果。为实现此目标,我们在instant-NGP的哈希编码模块中引入连续且可优化的内在坐标,替代原有的显式欧几里得坐标。借助这种新颖的内在坐标,IntrinsicNGP能够通过代理几何形状聚合动态对象的帧间信息。此外,基于给定粗略几何形状训练的结果,可通过基于内在坐标的可优化偏移场进行进一步细化。多个数据集上的大量实验结果表明了IntrinsicNGP的有效性和高效性。我们还展示了该方法对重建对象形状进行编辑的能力。