In recent years, Neural Fields (NFs) have emerged as an effective tool for encoding diverse continuous signals such as images, videos, audio, and 3D shapes. When applied to 3D data, NFs offer a solution to the fragmentation and limitations associated with prevalent discrete representations. However, given that NFs are essentially neural networks, it remains unclear whether and how they can be seamlessly integrated into deep learning pipelines for solving downstream tasks. This paper addresses this research problem and introduces nf2vec, a framework capable of generating a compact latent representation for an input NF in a single inference pass. We demonstrate that nf2vec effectively embeds 3D objects represented by the input NFs and showcase how the resulting embeddings can be employed in deep learning pipelines to successfully address various tasks, all while processing exclusively NFs. We test this framework on several NFs used to represent 3D surfaces, such as unsigned/signed distance and occupancy fields. Moreover, we demonstrate the effectiveness of our approach with more complex NFs that encompass both geometry and appearance of 3D objects such as neural radiance fields.

翻译：近年来，神经场作为一种有效编码图像、视频、音频和3D形状等多样连续信号的工具而兴起。当应用于3D数据时，神经场为当前离散表示方法的分割性与局限性提供了解决方案。然而，由于神经场本质上是神经网络，它们能否以及如何无缝集成到深度学习流程中以解决下游任务仍不明确。本文针对这一研究问题，提出了一种名为nf2vec的框架，该框架能在单次推理过程中为输入神经场生成紧凑的潜在表示。我们证明nf2vec能有效嵌入输入神经场所代表的3D对象，并展示了由此产生的嵌入如何被应用于深度学习流程中，在仅处理神经场的情况下成功解决多种任务。我们在多个用于表示3D曲面的神经场（如无符号/有符号距离场和占据场）上测试了这一框架。此外，我们通过更复杂的神经场（如同时包含3D对象几何与外观的神经辐射场）验证了该方法的有效性。