Kernel ridge regression (KRR) is a fundamental computational tool, appearing in problems that range from computational chemistry to health analytics, with a particular interest due to its starring role in Gaussian process regression. However, full KRR solvers are challenging to scale to large datasets: both direct (i.e., Cholesky decomposition) and iterative methods (i.e., PCG) incur prohibitive computational and storage costs. The standard approach to scale KRR to large datasets chooses a set of inducing points and solves an approximate version of the problem, inducing points KRR. However, the resulting solution tends to have worse predictive performance than the full KRR solution. In this work, we introduce a new solver, ASkotch, for full KRR that provides better solutions faster than state-of-the-art solvers for full and inducing points KRR. ASkotch is a scalable, accelerated, iterative method for full KRR that provably obtains linear convergence. Under appropriate conditions, we show that ASkotch obtains condition-number-free linear convergence. This convergence analysis rests on the theory of ridge leverage scores and determinantal point processes. ASkotch outperforms state-of-the-art KRR solvers on a testbed of 23 large-scale KRR regression and classification tasks derived from a wide range of application domains, demonstrating the superiority of full KRR over inducing points KRR. Our work opens up the possibility of as-yet-unimagined applications of full KRR across a number of disciplines.

翻译：核岭回归（KRR）是一种基础计算工具，其应用范围从计算化学延伸至健康分析领域，并因其在高斯过程回归中的核心作用而备受关注。然而，完整的KRR求解器难以扩展至大规模数据集：无论是直接方法（如Cholesky分解）还是迭代方法（如PCG）都会产生难以承受的计算和存储成本。将KRR扩展至大规模数据集的标准方法是选取一组诱导点并求解问题的近似版本，即诱导点KRR。然而，所得解往往比完整KRR解的预测性能更差。本文提出了一种新型求解器ASkotch，用于完整KRR问题，其求解速度优于当前最先进的完整KRR及诱导点KRR求解器，且能获得更优解。ASkotch是一种可扩展的加速迭代方法，用于完整KRR问题，并理论保证获得线性收敛性。在适当条件下，我们证明ASkotch能够实现与条件数无关的线性收敛。该收敛性分析基于岭杠杆得分与行列式点过程理论。在涵盖广泛应用领域的23个大规模KRR回归与分类任务测试集上，ASkotch的表现均优于当前最先进的KRR求解器，这证明了完整KRR相对于诱导点KRR的优越性。我们的工作为完整KRR在多个学科中尚未被设想的应用开辟了可能性。