Implicit neural representations (INRs) have demonstrated success in a variety of applications, including inverse problems and neural rendering. An INR is typically trained to capture one signal of interest, resulting in learned neural features that are highly attuned to that signal. Assumed to be less generalizable, we explore the aspect of transferability of such learned neural features for fitting similar signals. We introduce a new INR training framework, STRAINER that learns transferrable features for fitting INRs to new signals from a given distribution, faster and with better reconstruction quality. Owing to the sequential layer-wise affine operations in an INR, we propose to learn transferable representations by sharing initial encoder layers across multiple INRs with independent decoder layers. At test time, the learned encoder representations are transferred as initialization for an otherwise randomly initialized INR. We find STRAINER to yield extremely powerful initialization for fitting images from the same domain and allow for $\approx +10dB$ gain in signal quality early on compared to an untrained INR itself. STRAINER also provides a simple way to encode data-driven priors in INRs. We evaluate STRAINER on multiple in-domain and out-of-domain signal fitting tasks and inverse problems and further provide detailed analysis and discussion on the transferability of STRAINER's features. Our demo can be accessed at https://colab.research.google.com/drive/1fBZAwqE8C_lrRPAe-hQZJTWrMJuAKtG2?usp=sharing .
翻译:隐式神经表示(INRs)在逆问题和神经渲染等多种应用中已展现出成功。INR通常被训练以捕获一个感兴趣的信号,从而产生高度适应该信号的学习神经特征。尽管通常认为其泛化能力较弱,我们探索了此类学习神经特征在拟合相似信号时的可迁移性。我们提出了一种新的INR训练框架STRAINER,该框架学习可迁移特征,以更快速度、更高重建质量将INR拟合到给定分布中的新信号。基于INR中顺序的逐层仿射运算,我们提出通过共享多个具有独立解码器层的INR的初始编码器层来学习可迁移表示。在测试时,学习到的编码器表示可作为初始化参数迁移至原本随机初始化的INR。我们发现STRAINER能为同一域内的图像拟合提供极其强大的初始化,相比未经训练的INR本身,可在早期实现约+10dB的信号质量提升。STRAINER还提供了一种在INR中编码数据驱动先验的简单方法。我们在多个域内和域外信号拟合任务及逆问题上评估STRAINER,并进一步对STRAINER特征的可迁移性进行了详细分析和讨论。我们的演示可通过https://colab.research.google.com/drive/1fBZAwqE8C_lrRPAe-hQZJTWrMJuAKtG2?usp=sharing 访问。