We introduce an efficient optimization-based meta-learning technique for learning large-scale implicit neural representations (INRs). Our main idea is designing an online selection of context points, which can significantly reduce memory requirements for meta-learning in any established setting. By doing so, we expect additional memory savings which allows longer per-signal adaptation horizons (at a given memory budget), leading to better meta-initializations by reducing myopia and, more crucially, enabling learning on high-dimensional signals. To implement such context pruning, our technical novelty is three-fold. First, we propose a selection scheme that adaptively chooses a subset at each adaptation step based on the predictive error, leading to the modeling of the global structure of the signal in early steps and enabling the later steps to capture its high-frequency details. Second, we counteract any possible information loss from context pruning by minimizing the parameter distance to a bootstrapped target model trained on a full context set. Finally, we suggest using the full context set with a gradient scaling scheme at test-time. Our technique is model-agnostic, intuitive, and straightforward to implement, showing significant reconstruction improvements for a wide range of signals. Code is available at https://github.com/jihoontack/ECoP

翻译：我们提出了一种高效的基于优化的元学习技术，用于学习大规模隐式神经表示（INR）。核心思想是设计一种在线上下文点选择方法，该方法能够在任何现有设置下显著降低元学习的内存需求。通过这种方式，我们预期能进一步节省内存，从而在给定内存预算下延长每个信号的自适应周期，通过减少短视效应改善元初始化，更关键的是，使得高维信号的学习成为可能。为实现这种上下文剪枝，我们的技术新颖性体现在三个方面。首先，我们提出了一种基于预测误差的自适应选择子集方案，使得早期步骤能够建模信号的全局结构，后期步骤则能捕捉其高频细节。其次，通过最小化与在完整上下文集上训练的引导目标模型之间的参数距离，来抵消上下文剪枝可能造成的任何信息损失。最后，我们建议在测试阶段采用梯度缩放方案使用完整上下文集。我们的技术具有模型无关性、直观且易于实现，在广泛信号上展现了显著的信号重建改进。代码详见 https://github.com/jihoontack/ECoP