Muon updates matrix parameters via the matrix sign of the gradient and has shown strong empirical gains, yet its dynamics and scaling behavior remain unclear in theory. We study Muon in a linear associative memory model with softmax retrieval and a hierarchical frequency spectrum over query-answer pairs, with and without label noise. In this setting, we show that Gradient Descent (GD) learns frequency components at highly imbalanced rates, leading to slow convergence bottlenecked by low-frequency components. In contrast, the Muon optimizer mitigates this imbalance, leading to faster and more uniform progress. Specifically, in the noiseless case, Muon achieves an exponential speedup over GD; in the noisy case with a power-law frequency spectrum, we derive Muon's scaling law and demonstrate its superior scaling efficiency over GD. Furthermore, we show that Muon can be interpreted as an implicit matrix preconditioner arising from adaptive task alignment and block-symmetric gradient structure. In contrast, the preconditioner with coordinate-wise sign operator could match Muon under oracle access to unknown task representations, which is infeasible for SignGD in practice. Experiments on synthetic long-tail classification and LLaMA-style pre-training corroborate the theory.

翻译：Muon通过梯度的矩阵符号来更新矩阵参数，并展现出显著的实证效果，但其动力学和缩放行为在理论上仍不明确。我们在一个具有softmax检索和查询-答案对分层频谱的线性联想记忆模型中研究Muon，分别考虑有无标签噪声的情况。在此设定下，我们证明梯度下降（GD）以高度不均衡的速率学习频率分量，导致收敛缓慢且受限于低频分量。相比之下，Muon优化器缓解了这种不均衡，实现了更快且更均匀的进展。具体而言，在无噪声情况下，Muon相比GD实现了指数级加速；在具有幂律频谱的有噪声情况下，我们推导了Muon的缩放律，并证明了其相对于GD的优越缩放效率。此外，我们证明Muon可被解释为由自适应任务对齐和块对称梯度结构产生的隐式矩阵预处理子。相比之下，具有逐坐标符号运算的预处理子在获得任务表示的oracle访问时能匹配Muon，但这在SignGD实践中不可行。在合成长尾分类和LLaMA风格预训练上的实验验证了该理论。