Muon updates matrix parameters via the matrix sign of the gradient and has shown strong empirical gains, yet its dynamics and scaling behavior remain unclear in theory. We study Muon in a linear associative memory model with softmax retrieval and a hierarchical frequency spectrum over query-answer pairs, with and without label noise. In this setting, we show that Gradient Descent (GD) learns frequency components at highly imbalanced rates, leading to slow convergence bottlenecked by low-frequency components. In contrast, the Muon optimizer mitigates this imbalance, leading to faster and more uniform progress. Specifically, in the noiseless case, Muon achieves an exponential speedup over GD; in the noisy case with a power-decay frequency spectrum, we derive Muon's optimization scaling law and demonstrate its superior scaling efficiency over GD. Furthermore, we show that Muon can be interpreted as an implicit matrix preconditioner arising from adaptive task alignment and block-symmetric gradient structure. In contrast, the preconditioner with coordinate-wise sign operator could match Muon under oracle access to unknown task representations, which is infeasible for SignGD in practice. Experiments on synthetic long-tail classification and LLaMA-style pre-training corroborate the theory.

翻译：μon通过梯度的矩阵符号更新矩阵参数，并已展现出显著的实证优势，但其动力学特性和缩放行为在理论上仍不明确。本研究在线性联想记忆模型中考察μon优化器，该模型采用softmax检索机制，并在查询-回答对上构建层次化频率谱，同时考虑含标签噪声与无噪声两种场景。在此设定下，我们证明梯度下降法以高度不均衡的速率学习频率分量，导致收敛速度受低频分量制约而减缓。相比之下，μon优化器能有效缓解这种不均衡性，实现更快且更均匀的训练进程。具体而言，在无噪声情况下，μon相对梯度下降法获得指数级加速；在噪声环境且频率谱呈幂律衰减时，我们推导出μon的优化缩放定律，并证明其缩放效率显著优于梯度下降法。此外，我们揭示μon可解释为一种隐式矩阵预处理器，其源于自适应任务对齐机制和块对称梯度结构。与之对比，采用逐坐标符号算子的预处理器虽在理论上可通过未知任务表征的预言机访问匹配μon性能，但这对于实际中的SignGD并不可行。在合成长尾分类任务和LLaMA式预训练上的实验验证了理论结论。