We introduce the notion of an $\varepsilon$-cover for a kernel range space. A kernel range space concerns a set of points $X \subset \mathbb{R}^d$ and the space of all queries by a fixed kernel (e.g., a Gaussian kernel $K(p,\cdot) = \exp(-\|p-\cdot\|^2)$). For a point set $X$ of size $n$, a query returns a vector of values $R_p \in \mathbb{R}^n$, where the $i$th coordinate $(R_p)_i = K(p,x_i)$ for $x_i \in X$. An $\varepsilon$-cover is a subset of points $Q \subset \mathbb{R}^d$ so for any $p \in \mathbb{R}^d$ that $\frac{1}{n} \|R_p - R_q\|_1\leq \varepsilon$ for some $q \in Q$. This is a smooth analog of Haussler's notion of $\varepsilon$-covers for combinatorial range spaces (e.g., defined by subsets of points within a ball query) where the resulting vectors $R_p$ are in $\{0,1\}^n$ instead of $[0,1]^n$. The kernel versions of these range spaces show up in data analysis tasks where the coordinates may be uncertain or imprecise, and hence one wishes to add some flexibility in the notion of inside and outside of a query range. Our main result is that, unlike combinatorial range spaces, the size of kernel $\varepsilon$-covers is independent of the input size $n$ and dimension $d$. We obtain a bound of $(1/\varepsilon)^{\tilde O(1/\varepsilon^2)}$, where $\tilde{O}(f(1/\varepsilon))$ hides log factors in $(1/\varepsilon)$ that can depend on the kernel. This implies that by relaxing the notion of boundaries in range queries, eventually the curse of dimensionality disappears, and may help explain the success of machine learning in very high-dimensions. We also complement this result with a lower bound of almost $(1/\varepsilon)^{\Omega(1/\varepsilon)}$, showing the exponential dependence on $1/\varepsilon$ is necessary.

翻译：我们引入了核范围空间的$\varepsilon$-覆盖概念。核范围空间涉及点集$X \subset \mathbb{R}^d$以及由固定核（例如高斯核$K(p,\cdot) = \exp(-\|p-\cdot\|^2)$）定义的所有查询空间。对于规模为$n$的点集$X$，一次查询返回一个值向量$R_p \in \mathbb{R}^n$，其中第$i$个坐标$(R_p)_i = K(p,x_i)$对应$x_i \in X$。$\varepsilon$-覆盖是点集$Q \subset \mathbb{R}^d$的一个子集，使得对于任意$p \in \mathbb{R}^d$，存在某个$q \in Q$满足$\frac{1}{n} \|R_p - R_q\|_1\leq \varepsilon$。这是Haussler关于组合范围空间（例如由球查询内点子集定义的空间）中$\varepsilon$-覆盖概念的平滑类比，后者生成的向量$R_p$属于$\{0,1\}^n$而非$[0,1]^n$。这些范围空间的核版本出现在数据分​​析任务中，其中坐标可能具有不确定性或不精确性，因此需要在查询范围的“内部”与“外部”概念中引入一定灵活性。我们的主要结果表明，与组合范围空间不同，核$\varepsilon$-覆盖的规模与输入规模$n$和维度$d$无关。我们得到了一个$(1/\varepsilon)^{\tilde O(1/\varepsilon^2)}$的上界，其中$\tilde{O}(f(1/\varepsilon))$隐藏了可能依赖于核的$(1/\varepsilon)$的对数因子。这意味着，通过放宽范围查询中的边界概念，“维度灾难”最终消失，这或许能解释机器学习在高维空间中的成功。此外，我们通过一个几乎为$(1/\varepsilon)^{\Omega(1/\varepsilon)}$的下界补充了这一结果，表明对$1/\varepsilon$的指数依赖是必要的。