Neural Radiance Fields (NeRFs) have emerged as a groundbreaking paradigm for representing 3D objects and scenes by encoding shape and appearance information into the weights of a neural network. Recent studies have demonstrated that these weights can be used as input for frameworks designed to address deep learning tasks; however, such frameworks require NeRFs to adhere to a specific, predefined architecture. In this paper, we introduce the first framework capable of processing NeRFs with diverse architectures and performing inference on architectures unseen at training time. We achieve this by training a Graph Meta-Network within an unsupervised representation learning framework, and show that a contrastive objective is conducive to obtaining an architecture-agnostic latent space. In experiments conducted across 13 NeRF architectures belonging to three families (MLPs, tri-planes, and, for the first time, hash tables), our approach demonstrates robust performance in classification, retrieval, and language tasks involving multiple architectures, even unseen at training time, while also matching or exceeding the results of existing frameworks limited to single architectures. Our code and data are available at https://cvlab-unibo.github.io/gmnerf.
翻译:神经辐射场(NeRF)作为一种突破性范式,通过将形状和外观信息编码到神经网络的权重中,实现了对三维物体与场景的表征。近期研究表明,这些权重可作为专门处理深度学习任务的框架的输入;然而,此类框架要求NeRF必须遵循特定预定义的架构。本文提出了首个能够处理多样化架构NeRF,并在训练时未见过的架构上进行推理的框架。我们通过在无监督表示学习框架中训练图元网络实现这一目标,并证明对比学习目标有助于获得与架构无关的潜在空间。在涵盖三大类别(MLP、三平面及首次引入的哈希表)共13种NeRF架构的实验中,我们的方法在涉及多种架构(包括训练时未见过的架构)的分类、检索和语言任务中展现出鲁棒性能,同时达到或超越了现有局限于单一架构框架的结果。我们的代码与数据公开于https://cvlab-unibo.github.io/gmnerf。