We compute precise asymptotic expressions for the learning curves of least squares random feature (RF) models with either a separable strongly convex regularization or the $\ell_1$ regularization. We propose a novel multi-level application of the convex Gaussian min max theorem (CGMT) to overcome the traditional difficulty of finding computable expressions for random features models with correlated data. Our result takes the form of a computable 4-dimensional scalar optimization. In contrast to previous results, our approach does not require solving an often intractable proximal operator, which scales with the number of model parameters. Furthermore, we extend the universality results for the training and generalization errors for RF models to $\ell_1$ regularization. In particular, we demonstrate that under mild conditions, random feature models with elastic net or $\ell_1$ regularization are asymptotically equivalent to a surrogate Gaussian model with the same first and second moments. We numerically demonstrate the predictive capacity of our results, and show experimentally that the predicted test error is accurate even in the non-asymptotic regime.

翻译：我们计算了具有可分离强凸正则化或ℓ1正则化的最小二乘随机特征模型的学习曲线的精确渐近表达式。为克服传统上在关联数据下寻找随机特征模型可计算表达式的困难，我们提出了凸高斯极小极大定理的多层新型应用。我们的结果表现为一个可计算的4维标量优化形式。与以往结果相比，本方法无需求解通常难以处理的、与模型参数数量成正比的近端算子。此外，我们将随机特征模型训练误差与泛化误差的普适性结果推广至ℓ1正则化。具体而言，我们证明在温和条件下，采用弹性网络或ℓ1正则化的随机特征模型渐近等价于一个具有相同一阶矩和二阶矩的高斯替代模型。我们通过数值实验验证了结果预测能力，并实验表明即使在非渐近区域，预测的测试误差仍保持精确。