Kernel ridge regression (KRR) is widely used for nonparametric regression over reproducing kernel Hilbert spaces. It offers powerful modeling capabilities at the cost of significant computational costs, which typically require $O(n^3)$ computational time and $O(n^2)$ storage space, with the sample size n. We introduce a novel framework of multi-layer kernel machines that approximate KRR by employing a multi-layer structure and random features, and study how the optimal number of random features and layer sizes can be chosen while still preserving the minimax optimality of the approximate KRR estimate. For various classes of random features, including those corresponding to Gaussian and Matern kernels, we prove that multi-layer kernel machines can achieve $O(n^2\log^2n)$ computational time and $O(n\log^2n)$ storage space, and yield fast and minimax optimal approximations to the KRR estimate for nonparametric regression. Moreover, we construct uncertainty quantification for multi-layer kernel machines by using conformal prediction techniques with robust coverage properties. The analysis and theoretical predictions are supported by simulations and real data examples.

翻译：核岭回归（KRR）广泛应用于再生核希尔伯特空间上的非参数回归。它在样本量为n时，以显著的计算成本提供强大的建模能力，通常需要$O(n^3)$的计算时间和$O(n^2)$的存储空间。我们提出了一种全新的多层核机器框架，该框架通过采用多层结构和随机特征来近似KRR，并研究了如何在保持近似KRR估计的极小化最优性的同时，选择最优的随机特征数量和层级大小。对于包括高斯核和Matern核对应的各类随机特征，我们证明多层核机器能够实现$O(n^2\log^2n)$的计算时间和$O(n\log^2n)$的存储空间，并为非参数回归的KRR估计提供快速且极小化最优的近似。此外，我们利用具有稳健覆盖性质的共形预测技术，为多层核机器构建了不确定性量化方法。模拟实验和实际数据示例验证了本文的分析与理论预测。