Kernel methods provide an elegant framework for developing nonlinear learning algorithms from simple linear methods. Though these methods have superior empirical performance in several real data applications, their usefulness is inhibited by the significant computational burden incurred in large sample situations. Various approximation schemes have been proposed in the literature to alleviate these computational issues, and the approximate kernel machines are shown to retain the empirical performance. However, the theoretical properties of these approximate kernel machines are less well understood. In this work, we theoretically study the trade-off between computational complexity and statistical accuracy in Nystr\"om approximate kernel principal component analysis (KPCA), wherein we show that the Nystr\"om approximate KPCA matches the statistical performance of (non-approximate) KPCA while remaining computationally beneficial. Additionally, we show that Nystr\"om approximate KPCA outperforms the statistical behavior of another popular approximation scheme, the random feature approximation, when applied to KPCA.

翻译：内核方法为从简单的线性方法中开发非线性学习算法提供了一个优雅的框架。虽然这些方法在若干实际数据应用中具有优异的经验性,但由于在大量抽样情况下产生的重大计算负担,这些方法的效用受到阻碍。文献中提出了各种近似办法,以缓解这些计算问题,并显示近似内核机器保留了经验性能。然而,这些近似内核机器的理论特性不太为人所理解。在这项工作中,我们理论上研究了Nystr\\"om lobal 内核主部分分析(KPCA)中的计算复杂性和统计准确性之间的权衡。我们从中发现,Nystr\"lom 近似KPCA的统计性能(非近似)同时仍然具有计算效益。此外,我们表明,Nystr\"om lom 近似KPCA的统计性能比另一个流行的近似方案(随机地貌近似)的统计行为要好得多。