Recent work in random matrix theory (RMT) has developed the notion of deterministic equivalents: typically linear surrogate models that approximate the spectral behavior of large nonlinear random matrices, such as nonlinear feature maps in neural networks (NNs). Such equivalents make theoretical predictions tractable by reducing a complex model to a simpler one with properties that fall under the umbrella of classical RMT tools. However, this leaves open the question of whether this idealized linear equivalence remains meaningful for classification of high-dimensional nonlinearly separable data. Motivated by this, we consider the conjugate kernel (CK), which is the nonlinear feature map of a one-layer feedforward NN, under a canonical nonlinearly separable dataset for the XOR problem; and we use the study of informative outlier eigenvalues in the CK and whether their corresponding eigenvectors asymptotically align with XOR labels as a proxy for nonlinear learnability. We develop a robust quadratic equivalent of the CK matrix that enables a precise analysis of emergent informative spikes, as one modifies various knobs common in ML practice: sample complexity, signal-to-noise ratio (SNR), nonlinear activation choice, and pretrained features. We identify regimes in which these knobs move the CK beyond the linear equivalent and produce BBP-type transitions to label-aligned outlier eigenspaces. Our analysis helps bring deterministic-equivalence tools from RMT to bear on problems of practical relevance in ML.

翻译：近期随机矩阵理论（RMT）研究提出了确定性等价的概念：典型做法是用线性替代模型逼近大型非线性随机矩阵（如神经网络中的非线性特征映射）的谱行为。这种等价性通过将复杂模型简化为具有经典RMT工具特性的简化模型，使得理论预测变得易于处理。然而，这种方法未解决一个关键问题：这种理想化的线性等价是否能有效描述高维非线性可分数据的分类问题。受此启发，我们考虑单层前馈神经网络的共轭核（CK）——即其对异或（XOR）问题的典型非线性可分数据集的非线性特征映射；通过研究CK中具有信息量的异常特征值及其对应特征向量是否渐近对齐XOR标签，作为非线性可学习性的代理指标。我们建立了CK矩阵的稳健二次等价表征，能够精确分析当调整机器学习实践中的常见参数（样本复杂度、信噪比、非线性激活函数选择及预训练特征）时涌现的信息尖峰。我们确定了这些参数使CK超越线性等价并产生BBP型相变、导向标签对齐异常特征空间的参数区间。本文的分析有助于将RMT中的确定性等价工具应用于机器学习领域具有实际意义的问题。