The method of random Fourier features (RFF), proposed in a seminal paper by Rahimi and Recht (NIPS'07), is a powerful technique to find approximate low-dimensional representations of points in (high-dimensional) kernel space, for shift-invariant kernels. While RFF has been analyzed under various notions of error guarantee, the ability to preserve the kernel distance with \emph{relative} error is less understood. We show that for a significant range of kernels, including the well-known Laplacian kernels, RFF cannot approximate the kernel distance with small relative error using low dimensions. We complement this by showing as long as the shift-invariant kernel is analytic, RFF with $\mathrm{poly}(\epsilon^{-1} \log n)$ dimensions achieves $\epsilon$-relative error for pairwise kernel distance of $n$ points, and the dimension bound is improved to $\mathrm{poly}(\epsilon^{-1}\log k)$ for the specific application of kernel $k$-means. Finally, going beyond RFF, we make the first step towards data-oblivious dimension-reduction for general shift-invariant kernels, and we obtain a similar $\mathrm{poly}(\epsilon^{-1} \log n)$ dimension bound for Laplacian kernels. We also validate the dimension-error tradeoff of our methods on simulated datasets, and they demonstrate superior performance compared with other popular methods including random-projection and Nystr\"{o}m methods.

翻译：随机傅里叶特征（RFF）方法由Rahimi和Recht在其开创性论文（NIPS'07）中提出，是一种为（高维）核空间中点寻找近似低维表示的强大技术，适用于平移不变核。尽管RFF已在多种误差保证概念下得到分析，但其在保持核距离的*相对*误差方面的能力尚不明确。我们证明，对于包括著名拉普拉斯核在内的一类重要核，RFF无法通过低维度以较小相对误差近似核距离。作为补充，我们进一步证明：只要平移不变核是解析的，使用$\mathrm{poly}(\epsilon^{-1} \log n)$维度的RFF即可对$n$个点的成对核距离实现$\epsilon$相对误差；对于核$k$均值这一特定应用，维度界可改进至$\mathrm{poly}(\epsilon^{-1}\log k)$。最后，超越RFF，我们首次探索了一般平移不变核的数据无关降维方法，并对拉普拉斯核获得了类似的$\mathrm{poly}(\epsilon^{-1} \log n)$维度界。我们还在模拟数据集上验证了所提方法的维度-误差权衡，其性能优于随机投影和Nyström等其他流行方法。