Neural fields, a category of neural networks trained to represent high-frequency signals, have gained significant attention in recent years due to their impressive performance in modeling complex 3D data, especially large neural signed distance (SDFs) or radiance fields (NeRFs) via a single multi-layer perceptron (MLP). However, despite the power and simplicity of representing signals with an MLP, these methods still face challenges when modeling large and complex temporal signals due to the limited capacity of MLPs. In this paper, we propose an effective approach to address this limitation by incorporating temporal residual layers into neural fields, dubbed ResFields, a novel class of networks specifically designed to effectively represent complex temporal signals. We conduct a comprehensive analysis of the properties of ResFields and propose a matrix factorization technique to reduce the number of trainable parameters and enhance generalization capabilities. Importantly, our formulation seamlessly integrates with existing techniques and consistently improves results across various challenging tasks: 2D video approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF reconstruction. Lastly, we demonstrate the practical utility of ResFields by showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory inputs of a lightweight capture system.

翻译：神经场是一类用于表示高频信号的神经网络，近年来因其在建模复杂三维数据（尤其是通过单一多层感知器（MLP）表示大规模神经有符号距离函数（SDF）或辐射场（NeRF））方面的卓越性能而受到广泛关注。然而，尽管使用MLP表示信号具有强大功能和简洁性，但由于MLP容量有限，这类方法在建模大规模复杂时间信号时仍面临挑战。本文提出了一种有效方法来解决这一局限：将时间残差层引入神经场，称为ResFields——一种专为有效表示复杂时间信号而设计的新型网络类别。我们对ResFields的特性进行了全面分析，并提出了一种矩阵分解技术，以减少可训练参数数量并增强泛化能力。重要的是，我们的公式能与现有技术无缝集成，并在多项具有挑战性的任务中持续提升结果：二维视频近似、通过时间SDF进行动态形状建模以及动态NeRF重建。最后，我们通过展示ResFields在从轻量级采集系统的稀疏感知输入中捕获动态三维场景方面的有效性，验证了其实用价值。