Nonparametric models are of great interest in various scientific and engineering disciplines. Classical kernel methods, while numerically robust and statistically sound in low-dimensional settings, become inadequate in higher-dimensional settings due to the curse of dimensionality. In this paper, we introduce a new framework called Variance-Reduced Sketching (VRS), specifically designed to estimate density functions and nonparametric regression functions in higher dimensions with a reduced curse of dimensionality. Our framework conceptualizes multivariable functions as infinite-size matrices, and facilitates a new sketching technique motivated by numerical linear algebra literature to reduce the variance in estimation problems. We demonstrate the robust numerical performance of VRS through a series of simulated experiments and real-world data applications. Notably, VRS shows remarkable improvement over existing neural network estimators and classical kernel methods in numerous density estimation and nonparametric regression models. Additionally, we offer theoretical justifications for VRS to support its ability to deliver nonparametric estimation with a reduced curse of dimensionality.

翻译：非参数模型在各类科学与工程学科中备受关注。经典核方法在低维场景中具有数值稳健性和统计可靠性，但受维度灾难影响，在高维场景中表现不足。本文提出一种名为方差缩减草图（VRS）的新框架，专门用于在高维环境下以减轻维度灾难的方式估计密度函数与非参数回归函数。该框架将多元函数视为无限维矩阵，并借鉴数值线性代数文献中的思想发展出一种新型草图技术，以降低估计问题中的方差。我们通过一系列模拟实验和真实数据应用验证了VRS的稳健数值性能。值得注意的是，在众多密度估计与非参数回归模型中，VRS相较于现有神经网络估计器和经典核方法展现出显著改进。此外，我们为VRS提供了理论依据，以支撑其实现减轻维度灾难的非参数估计能力。