Neural fields, a category of neural networks trained to represent high-frequency signals, have gained significant attention in recent years due to their impressive performance in modeling complex 3D data, especially large neural signed distance (SDFs) or radiance fields (NeRFs) via a single multi-layer perceptron (MLP). However, despite the power and simplicity of representing signals with an MLP, these methods still face challenges when modeling large and complex temporal signals due to the limited capacity of MLPs. In this paper, we propose an effective approach to address this limitation by incorporating temporal residual layers into neural fields, dubbed ResFields, a novel class of networks specifically designed to effectively represent complex temporal signals. We conduct a comprehensive analysis of the properties of ResFields and propose a matrix factorization technique to reduce the number of trainable parameters and enhance generalization capabilities. Importantly, our formulation seamlessly integrates with existing techniques and consistently improves results across various challenging tasks: 2D video approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF reconstruction. Lastly, we demonstrate the practical utility of ResFields by showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory inputs of a lightweight capture system.

翻译：神经场是一类训练用于表示高频信号的神经网络，近年来因其在建模复杂三维数据（尤其是通过单个多层感知机（MLP）表示大型神经有符号距离函数（SDF）或辐射场（NeRF））中的卓越性能而受到广泛关注。然而，尽管使用MLP表示信号具有强大和简洁的优势，但由于MLP的容量有限，这些方法在建模大型复杂时变信号时仍面临挑战。本文提出了一种有效方法来解决这一局限，通过将时间残差层引入神经场，构建了名为ResFields的新型网络类别，专为高效表示复杂时变信号而设计。我们对ResFields的特性进行了全面分析，并提出了一种矩阵分解技术来减少可训练参数数量并增强泛化能力。重要的是，我们的公式能够无缝集成现有技术，并在多项具有挑战性的任务中持续改进结果：二维视频近似、基于时变SDF的动态形状建模以及动态NeRF重建。最后，我们通过展示ResFields在从轻量级采集系统的稀疏传感输入中捕获动态三维场景的有效性，验证了其实用价值。