Implicit Neural Representations (INRs) based on vanilla Multi-Layer Perceptrons (MLPs) are widely believed to be incapable of representing high-frequency content. This has directed research efforts towards architectural interventions, such as coordinate embeddings or specialized activation functions, to represent high-frequency signals. In this paper, we challenge the notion that the low-frequency bias of vanilla MLPs is an intrinsic, architectural limitation to learn high-frequency content, but instead a symptom of stable rank degradation during training. We empirically demonstrate that regulating the network's rank during training substantially improves the fidelity of the learned signal, rendering even simple MLP architectures expressive. Extensive experiments show that using optimizers like Muon, with high-rank, near-orthogonal updates, consistently enhances INR architectures even beyond simple ReLU MLPs. These substantial improvements hold across a diverse range of domains, including natural and medical images and novel view synthesis, with up to +9 dB PSNR over the same architecture. Code is available at (https://rank-inrs.github.io).

翻译：基于原始多层感知器（MLP）的隐式神经表示（INR）普遍被认为无法表示高频内容。这一认知推动了研究转向架构改进（如坐标嵌入或专用激活函数）以表示高频信号。本文质疑了"原始MLP的低频偏好是其学习高频内容的固有架构限制"这一观点，提出这实际上是训练过程中稳定秩退化的表现。我们通过实验证明，在训练过程中调控网络的秩能显著提升学习信号的质量，即使简单的MLP架构也能具备高表达能力。大量实验表明，采用Muon等具有高秩、近正交更新的优化器，能持续增强INR架构性能，甚至超越简单的ReLU型MLP。这些显著改进在自然图像、医学图像和新视角合成等多个领域均成立，在相同架构下PSNR提升最高达9 dB。代码开源地址：(https://rank-inrs.github.io)。