We provide exact asymptotic expressions for the performance of regression by an $L-$layer deep random feature (RF) model, where the input is mapped through multiple random embedding and non-linear activation functions. For this purpose, we establish two key steps: First, we prove a novel universality result for RF models and deterministic data, by which we demonstrate that a deep random feature model is equivalent to a deep linear Gaussian model that matches it in the first and second moments, at each layer. Second, we make use of the convex Gaussian Min-Max theorem multiple times to obtain the exact behavior of deep RF models. We further characterize the variation of the eigendistribution in different layers of the equivalent Gaussian model, demonstrating that depth has a tangible effect on model performance despite the fact that only the last layer of the model is being trained.

翻译：我们提供了$L$层深度随机特征（RF）模型回归性能的精确渐近表达式，其中输入通过多个随机嵌入和非线性激活函数进行映射。为此，我们建立了两个关键步骤：首先，我们证明了RF模型与确定性数据的一个新普适性结果，由此证明深度随机特征模型等价于一个在每一层的一阶矩和二阶矩上与之匹配的深度线性高斯模型。其次，我们多次利用凸高斯极小极大定理获取深度RF模型的精确行为。我们进一步刻画了等效高斯模型不同层中特征分布的变化，证明尽管仅训练模型的最后一层，深度对模型性能仍有显著影响。